-- Logs begin at Tue 2022-07-05 09:25:06 UTC, end at Tue 2022-07-05 10:38:34 UTC. -- Jul 05 10:01:47.487504 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jul 05 10:01:47.495576 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 05 10:01:47.498051 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 5 10:01:47 2022] *** Jul 05 10:01:47.498051 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 05 10:01:47.498051 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 05 10:01:47.498051 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: nodename: dv-cinder-master-3-iscsi Jul 05 10:01:47.498051 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: machine: x86_64 Jul 05 10:01:47.498051 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: clock source: unix Jul 05 10:01:47.498051 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: pcre jit disabled Jul 05 10:01:47.498051 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: detected number of CPU cores: 4 Jul 05 10:01:47.498051 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: current working directory: / Jul 05 10:01:47.498051 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: detected binary path: /usr/bin/uwsgi-core Jul 05 10:01:47.498634 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: your processes number limit is 47680 Jul 05 10:01:47.498634 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: your memory page size is 4096 bytes Jul 05 10:01:47.498634 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: detected max file descriptor number: 2048 Jul 05 10:01:47.498634 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: lock engine: pthread robust mutexes Jul 05 10:01:47.498634 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: thunder lock: enabled Jul 05 10:01:47.498634 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 05 10:01:47.498634 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 05 10:01:47.541673 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: Python main interpreter initialized at 0x55f5054459c0 Jul 05 10:01:47.541673 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: python threads support enabled Jul 05 10:01:47.541673 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: your server socket listen backlog is limited to 100 connections Jul 05 10:01:47.541673 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: your mercy for graceful operations on workers is 90 seconds Jul 05 10:01:47.542015 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: mapped 403077 bytes (393 KB) for 2 cores Jul 05 10:01:47.542088 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: *** Operational MODE: preforking *** Jul 05 10:01:47.542337 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: *** uWSGI is running in multiple interpreter mode *** Jul 05 10:01:47.542337 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: spawned uWSGI master process (pid: 103282) Jul 05 10:01:47.542536 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jul 05 10:01:47.543588 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: spawned uWSGI worker 1 (pid: 103283, cores: 1) Jul 05 10:01:47.543588 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: spawned uWSGI worker 2 (pid: 103284, cores: 1) Jul 05 10:01:47.543588 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 05 10:01:48.191953 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 05 10:01:48.196900 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 05 10:01:49.300300 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 10:01:49.300300 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: from cryptography.utils import int_from_bytes Jul 05 10:01:49.300903 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 10:01:49.300903 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: from cryptography.utils import int_from_bytes Jul 05 10:01:49.304875 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 10:01:49.304875 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: from cryptography.utils import int_from_bytes Jul 05 10:01:49.305285 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 10:01:49.305285 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: from cryptography.utils import int_from_bytes Jul 05 10:01:51.786398 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 05 10:01:51.786975 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 05 10:01:51.787150 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 05 10:01:51.787511 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 05 10:01:51.787764 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 05 10:01:51.788146 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.788376 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 05 10:01:51.788754 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.788853 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 05 10:01:51.788942 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.789106 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.789208 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 05 10:01:51.789430 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.789530 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 05 10:01:51.789735 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.789833 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 05 10:01:51.790053 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.790150 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.790368 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.790460 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.790690 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.790785 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.790946 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.791037 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.791258 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.791356 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.791581 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.791674 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.791897 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.792044 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.792206 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.792328 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.792538 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.792642 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.792854 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.792958 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.793152 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.793266 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.793472 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.793567 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.793774 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.793875 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.794090 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.794264 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.794416 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.794701 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.794880 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.795031 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.795176 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.795322 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.795466 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.795654 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.795752 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.795911 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.796100 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.796204 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.796429 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.796605 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.796739 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.796832 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.797050 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.797137 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.797388 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.797487 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.797701 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.797788 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.798008 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.798106 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.798328 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.798415 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.798636 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.798729 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.798959 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.799090 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.799269 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.799400 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.799662 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.799754 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.799976 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.800079 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.800287 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.800380 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.800634 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.800735 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.800979 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.801064 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.801298 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.801393 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.801618 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.801710 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.801933 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.802024 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.802231 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.802394 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.802536 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.802629 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.802840 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.802931 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.803153 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.803247 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.803453 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.803543 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.803752 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.803848 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.803999 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.804092 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.804303 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.804415 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.804645 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.804746 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.804961 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.805052 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.805277 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.805373 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.805527 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.805620 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.805773 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.805861 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.806087 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.806173 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.806401 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.806488 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.806706 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.806799 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.807008 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.807094 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.807289 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.807382 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.807542 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.807636 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.807864 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.807959 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.808180 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.808273 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.808507 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.808596 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.808816 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.808916 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.809122 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.809207 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.809431 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.809525 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.809736 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.809822 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.810042 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.810126 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.810348 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.810439 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.810658 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.810744 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.20 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.810974 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.20 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.811059 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.811278 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.811364 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.811601 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.811687 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.811914 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.812006 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.812216 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.812308 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.812564 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.812713 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.20 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.812882 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.812973 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.20 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.813187 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.813280 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.813491 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.813577 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.813793 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.813900 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.814108 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.814204 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.814436 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.814522 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.814735 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.814828 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.814983 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.815119 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.815280 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.815426 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.815581 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.815734 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.815894 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.816055 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.816198 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.816924 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.817030 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.817237 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.817330 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.817490 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.817622 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.817792 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.817927 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.818099 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.818215 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.818420 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.818513 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.818729 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.818820 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.819038 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.819147 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.819347 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.819435 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.819646 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.819733 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.819952 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.820047 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.820250 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.820335 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.820581 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.820675 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.820890 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.820986 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.821191 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.821277 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.821494 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.821588 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.821796 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.821889 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.822101 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.822196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.822406 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.822504 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.822708 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.822809 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.823014 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.823100 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.823322 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.823407 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.823622 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.823708 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.823923 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.824009 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.824229 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.824320 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.824555 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.824656 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.824860 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.824980 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.825165 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.825250 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.825459 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.825552 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.825762 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.825847 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.826071 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.826155 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.826374 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.826465 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.826677 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.826764 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.826976 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.827071 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.827274 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.827366 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.827574 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.827660 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.827875 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.827962 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.828172 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.828261 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.828495 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.828585 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.828793 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.828890 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.829095 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.829185 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.829400 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.829487 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.829704 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.829787 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.830005 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.830079 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.830305 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.830401 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.830621 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.830720 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.830919 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.831015 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.831225 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.831317 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.831548 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.831642 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.831857 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.831945 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.832726 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.832825 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.833039 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.833128 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.833359 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:01:51.833451 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.833667 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.833807 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.833981 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.834069 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.834303 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.834389 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.834611 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.834696 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.834919 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.835005 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.835226 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.835311 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.835541 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.835640 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.835856 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.835943 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.836180 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.836266 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.836516 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.836618 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.836832 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.836917 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.837145 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.837240 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.837450 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.837535 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.837766 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.837858 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.838081 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.838211 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.838396 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.838490 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.838704 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.838791 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.839021 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.839141 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.839342 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.839442 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.839655 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.839741 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.839971 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.840056 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.840283 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.840416 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.840628 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.840721 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.840933 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.841019 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.841237 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.841331 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.841551 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.841672 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.841886 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.841984 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.842194 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.842280 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.842506 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.842592 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.842812 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.842905 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.843119 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.843206 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.843422 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.843515 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.843728 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.843850 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.844033 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.844128 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.844363 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.844502 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.844694 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.844781 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.845007 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.845106 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.845313 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.845398 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.845621 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.845720 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.845926 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.846018 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.846232 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.846317 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.846537 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.846640 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.846836 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.846927 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.847134 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.847220 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.847431 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.847529 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.847757 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.847842 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.848055 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.848140 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.848370 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.848534 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.848716 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.848803 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.849018 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.849102 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.849324 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.849424 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.849627 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.849713 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.849932 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.850030 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.850234 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.850320 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.850537 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.850621 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.850838 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.850931 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.851144 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.851231 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.851449 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.851535 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.851748 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.851832 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.852061 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.852154 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.852364 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.852478 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.852688 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.852775 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.852991 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.853077 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.853290 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.853375 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.853587 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.853682 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.853894 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.853987 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.854198 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.854283 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.854501 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.854600 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.854804 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.854888 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.855104 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.855191 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.855415 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.855508 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.855719 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.855805 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.856022 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.856108 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.856328 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.856443 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.856666 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.856760 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.856974 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.857059 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.857282 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.857368 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.857598 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.857689 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.857907 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.857992 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.858204 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.858291 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.858505 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.858591 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.858804 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.858890 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.859103 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.859190 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.859403 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.859489 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.859703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.859788 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.860000 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.860087 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.860298 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.860427 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.860621 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.860726 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.860918 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.861003 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.861216 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.861301 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.861530 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.861624 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.861831 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.861929 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.862132 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.862217 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.862426 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.862524 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.862723 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.862809 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.863024 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.863108 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.863352 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.863449 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.863680 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.863766 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.863986 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.864117 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.864291 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.864378 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.864617 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.864716 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.864923 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.865009 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.865230 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.865316 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.865536 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.865622 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.865856 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.865965 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.866146 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.866233 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.866450 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.866536 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.866764 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.866854 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.867068 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.867155 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.867379 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.867465 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.867683 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.867787 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.867998 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.868082 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.868300 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.868408 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.868635 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.868730 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.868931 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.869017 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.869239 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.869325 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.869561 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.869656 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.869873 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.869960 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.870181 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.870281 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.870492 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.870585 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.870794 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.870886 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.871100 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.871185 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.871402 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.871495 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.871706 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.871792 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.872015 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.872101 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.872320 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.872426 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.872659 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.872755 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.872964 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.873049 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.873276 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.873372 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.873578 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.873664 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.873886 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.873976 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.874189 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.874281 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.874499 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.874584 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.874792 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.874876 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.875099 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.875187 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.875418 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.875503 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.875726 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.875823 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.876032 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.876118 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.876341 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.876476 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.876680 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.876786 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.876982 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.877078 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.877284 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.877377 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.877591 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.877685 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.877890 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.877976 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.878188 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.878275 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.878492 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.878634 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.878800 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.879396 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.879636 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.879730 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.879941 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.880028 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.880239 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.880325 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.880568 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.880654 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.880872 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.880958 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.881170 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.881265 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.881468 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.881561 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.881773 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.881866 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.882081 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.882165 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.882385 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.882471 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.882696 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.882782 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.883002 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.883093 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.883301 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.883435 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.883599 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.883686 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.883907 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.883991 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.884209 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.884306 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.884533 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.884622 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.884834 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.884921 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.885132 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.885227 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.885466 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.885552 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.885768 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.885860 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.886067 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.886162 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.886377 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.886463 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.886679 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.886764 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.886981 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.887066 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.887288 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.887402 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.887590 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.887685 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.887893 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.887980 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.888207 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.888299 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.888537 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.888627 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.888843 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.888937 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.889148 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.889241 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.889454 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.889539 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.889759 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.889846 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.890063 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.890148 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.890378 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.890471 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.890682 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.890768 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.890986 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.891071 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.891289 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.891382 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.891589 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.891675 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.891894 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.891980 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.892203 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.892300 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.892561 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.892791 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.892877 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.893120 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.893217 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.893432 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.893558 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.893747 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.893843 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.894061 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.894156 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.894376 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.894462 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.895250 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.895492 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.895579 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.895838 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.895926 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.896162 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.896256 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.896502 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.896591 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.896805 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.896904 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.897105 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.897191 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.897411 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.897496 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.897713 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.897807 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.898013 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.898112 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.898313 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.898417 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.898618 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.898705 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.898917 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.899003 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.899215 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.899301 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.899514 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.899599 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.899817 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.899905 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.900127 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.900214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.900445 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.900546 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.900756 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.900842 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.901056 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.901149 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.901370 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.901464 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.901674 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.901761 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.901971 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.902056 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.902269 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.902372 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.902572 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.902658 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.902884 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.902969 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.903192 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.903289 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.903517 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.903603 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.903828 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.903921 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.904141 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.904237 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.904466 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.904567 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.904767 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.904853 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.905085 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.905173 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.905385 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.905472 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.905683 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.905776 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.905980 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.906068 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.906282 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.906376 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.906584 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.906681 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.907135 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.907235 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.907444 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.907530 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.907749 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.907837 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.908053 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.908147 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.908355 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.908470 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.908677 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.908867 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.908979 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.909278 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.909441 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.909587 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.910626 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.910723 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.910805 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.910889 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.910969 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.911053 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.911133 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.911219 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.911299 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.911376 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.911453 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.911529 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.911639 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.911739 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.911938 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.912034 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.912249 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.912335 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.912579 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.912668 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.912880 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.912973 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.913177 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.913264 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.913479 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.913608 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.913782 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.913870 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.914088 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.914176 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.914396 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.914484 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.914701 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.914794 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.914999 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.915084 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.915297 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.915390 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.915599 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.915692 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.915898 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.915993 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.916200 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.916294 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.916531 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.916637 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.916840 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.916940 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.917151 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.917235 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.917449 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.917534 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.917753 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.917847 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.918051 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.918143 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.918350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.918436 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.918975 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.919123 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.919233 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.919346 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.919452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.919556 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.919670 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.919775 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.919879 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.919983 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.920088 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.920192 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.920370 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.920502 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.920707 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.920821 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.921009 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.921124 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.921309 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.921430 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.921609 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.921723 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.921908 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.922021 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.922216 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.922330 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.922520 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.922634 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.922826 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.922954 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.923131 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.923245 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.923453 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.923566 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.923761 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.923884 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.924065 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.924177 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.924369 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.924512 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.924696 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.924823 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.924994 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.925106 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.925302 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.925425 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.925607 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.926303 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.926419 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.926524 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.926710 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.926837 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.927008 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.927136 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.927324 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.927446 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.927632 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.927745 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.927937 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.928059 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.928758 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.928858 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.928940 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.929026 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.929106 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.929190 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.929269 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.929346 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.929455 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.929551 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.929760 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.929846 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.930062 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.930149 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.930367 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.930505 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.930668 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.930755 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.930987 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.931076 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.931278 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.931373 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.931584 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.931671 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.931890 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.931975 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.932194 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.932282 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.932517 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.932620 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.932828 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.932923 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.933131 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.933219 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.933430 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.933523 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.933746 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.933831 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.934046 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.934139 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.934343 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.934431 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.934644 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.934739 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.934943 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.935030 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.935246 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.935332 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.935543 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.935641 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.935857 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.935964 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.936166 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.936252 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.936515 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.936619 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.936829 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.936924 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.937134 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.937221 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.937445 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.937539 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.937753 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.937838 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.938059 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.938145 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.938415 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.938515 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.938767 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.938860 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.939074 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.939161 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.939386 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.939473 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.939693 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.939779 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.940007 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.940094 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.940303 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.940410 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.940634 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.940722 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.940952 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.941038 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.941249 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.941335 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.942086 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.942225 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.942389 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.942547 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.942706 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.942866 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.943010 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.943184 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.943324 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.943492 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.943732 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.943818 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.944038 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.944124 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.944336 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.944469 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.944740 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.944825 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.945049 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.945136 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.945357 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.945452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.945673 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.945758 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.945975 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.946060 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.946296 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.946383 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.946598 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.946710 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.946908 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.947025 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.947212 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.947334 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.947530 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.947749 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.947846 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.948063 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.948242 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.948377 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.948617 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.948716 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.948798 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.949035 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.949130 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.949362 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.949464 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.949668 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.949754 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.949983 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.950068 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.950294 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.950389 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.950619 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.950706 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.950944 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.951030 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.951260 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.951358 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.951577 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.951664 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.951876 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.951963 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.952177 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.952265 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.952506 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.952615 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.952815 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.952902 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.953115 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.953204 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.953431 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.953547 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.953732 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.953873 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.954038 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.954188 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.954346 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.954516 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.954649 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.954843 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.954931 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.955151 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.955237 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.955458 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.955552 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.955771 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.955859 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.956080 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.956166 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.956397 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.956486 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.956731 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.956823 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.957071 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.957160 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.957373 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.957459 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.957691 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.957779 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.957993 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.958086 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.958294 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.958389 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.958599 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.958742 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.958901 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.958997 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.959209 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.959304 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.959518 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.959603 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.959833 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.959927 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.960154 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.960248 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.960497 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.960597 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.960804 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.960892 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.961127 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.961263 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.961438 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.961525 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.961746 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.961831 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.962050 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.962143 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.962358 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.962444 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.962653 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.962737 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.962955 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.963049 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.963256 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.963343 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.963554 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.963641 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.963850 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.963937 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.964147 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.964233 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.964473 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.964592 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.964772 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.964923 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.965068 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.965246 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.965363 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.965568 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.965669 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.965897 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.966031 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.966206 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.966304 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.966514 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.966602 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.966827 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.966913 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.967121 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.967205 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.967418 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.967505 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.967723 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.967808 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.968033 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.968119 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.968339 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.968481 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.968668 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.968770 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.968977 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.969085 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.969299 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.969389 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.969601 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.969687 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.969919 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.970014 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.970228 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.970313 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.970540 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.970659 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.970847 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.970993 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.971157 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.971439 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.971529 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.971760 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.971853 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.972070 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.972165 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.972372 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.972491 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.972729 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.972835 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.973041 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.973128 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.973350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.973436 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.973692 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.973880 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.974019 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.974203 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.974340 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.974513 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.974668 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.974821 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.974988 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.975128 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.975309 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.975438 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.975630 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.975747 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.975949 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.976057 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.976269 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.976357 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.976607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.976697 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.976927 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.977014 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.977246 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.977341 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.977565 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.977653 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.977888 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.978103 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.978220 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.978435 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.978530 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.978753 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.978891 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.979066 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.979207 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.979378 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.979482 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.979697 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.979784 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.980013 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.980107 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.980326 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.980434 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.980664 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.980751 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.980972 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.981059 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.981284 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.981372 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.981594 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.981693 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.981903 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.981998 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.982214 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.982301 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.982524 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.982611 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.982835 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.982934 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.983139 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.983226 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.983448 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.983544 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.983759 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.983848 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.984078 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.984166 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.984423 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.984555 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.984744 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.984832 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.985054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.985140 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.985363 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.985449 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.985676 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.985763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.985992 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.986190 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.986306 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.986500 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.986622 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.986829 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.986937 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.987151 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.987237 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.987471 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.987584 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.987781 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.987876 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.988095 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.988183 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.988961 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.989055 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.989269 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.989368 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.989568 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.989666 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.989871 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.989966 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.990175 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.990262 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.990475 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.990571 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.990791 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.991044 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.991131 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.991365 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.991452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.991684 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.991771 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.991988 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.992075 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.992330 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.992545 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.992677 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.992897 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.992983 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.993219 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.993305 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.993546 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.993634 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.993873 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.993960 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.994192 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.994277 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.994501 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.994596 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.994807 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.994901 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.995112 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.995198 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.995418 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.995506 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.995729 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.995815 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.996035 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.996121 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.996337 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.996451 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.996675 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.996893 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.996980 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.997241 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.997330 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.997561 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.997648 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.997865 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.997949 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.998169 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.998266 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.998493 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.998579 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.998797 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.998883 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.999100 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.999188 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.999405 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.999504 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.999709 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:51.999797 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.000010 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.000097 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.000310 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.000427 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.000637 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.000738 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.000939 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.001027 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.001248 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.001334 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.001547 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.001645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.001852 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.001940 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.002149 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.002236 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.002457 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.002555 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.002752 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.002838 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.003054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.003141 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.003374 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.003472 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.003678 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.003765 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.004550 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.004648 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.004854 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.004941 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.005164 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.005254 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.005467 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.005552 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.005770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.005863 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.006080 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.006174 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.006384 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.006470 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.006693 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.006781 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.006999 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.007087 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.007310 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.007529 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.007625 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.007860 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.007948 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.008180 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.008274 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.008545 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.008645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.008876 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.008963 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.009180 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.009266 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.009482 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.009587 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.009778 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.009866 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.010095 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.010181 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.010402 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.010496 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.010707 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.010795 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.011012 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.011098 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.011321 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.011409 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.011625 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.011877 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.011965 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.012214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.012303 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.012541 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.012632 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.012846 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.012932 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.013149 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.013238 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.013457 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.013544 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.013760 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.013852 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.014061 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.014156 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.014370 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.014457 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.014706 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.014805 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.015021 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.015115 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.015331 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.015416 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.015638 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.015736 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.015947 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.016032 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.016248 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.016333 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.016579 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.016680 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.016883 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.016968 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.017189 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.017282 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.017497 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.017592 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.017803 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.017889 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.018112 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.018198 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.018408 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.018501 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.018722 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.018808 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.019063 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.019149 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.019386 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.019474 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.019735 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.019834 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.020035 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.020129 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.020351 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.020472 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.020687 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.020773 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.020993 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.021096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.021300 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.021396 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.021603 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.021695 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.021919 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.022014 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.022249 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.022337 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.022562 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.022656 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.022869 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.022965 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.023168 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.023283 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.023472 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.023595 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.023772 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.023915 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.024075 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.024237 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.024404 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.024571 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.024735 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.024894 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.024982 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.025222 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.025321 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.025534 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.025630 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.025853 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.025940 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.026151 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.026244 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.026448 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.026543 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.026745 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.026832 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.027047 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.027135 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.027340 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.027428 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.027636 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.027723 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.027933 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.028020 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.028228 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.028317 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.028548 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.028648 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.028845 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.028932 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.029146 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.029234 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.029442 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.029528 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.029798 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.029903 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.030025 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.030113 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.030321 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.030414 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.030616 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.030711 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.030940 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.031027 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.031250 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.031335 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.031559 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.031655 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.032112 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.032206 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.032432 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.032523 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.032741 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.032841 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.033044 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.033145 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.033347 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.033424 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.033650 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.033744 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.033960 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.034047 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.034264 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.034352 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.034568 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.034657 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.034873 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.034959 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.035202 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.035291 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.035511 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.035599 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.035816 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.035902 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.036121 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.036219 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.036561 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.036656 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.036842 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.036929 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.037162 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.037263 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.037463 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.037552 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.037782 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.037879 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.038093 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.038188 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.038406 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.038492 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.038716 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.038802 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.039022 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.039117 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.039343 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.039430 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.039649 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.039740 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.039958 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.040079 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.040266 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.040367 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.040596 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.040696 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.040902 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.040993 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.041211 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.041297 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.041519 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.041606 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.041838 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.041935 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.042225 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.042324 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.042453 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.042555 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.042774 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.042862 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.043093 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.043180 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.043409 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.043507 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.043725 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.043814 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.044056 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.044143 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.044370 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.044493 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.044702 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.044790 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.045009 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.045095 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.045310 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.045398 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.045550 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 05 10:01:52.045639 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.045929 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.046249 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.046577 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.046906 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.047218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.047547 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.047874 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.048223 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.048561 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.048907 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.049236 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.049621 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.049932 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.050176 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:01:52.050421 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103283) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 05 10:01:52.106437 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO dbcounter [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] Registered counter for database nova_api Jul 05 10:01:52.111839 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO dbcounter [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Registered counter for database nova_api Jul 05 10:01:52.219066 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG oslo_db.sqlalchemy.engines [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103284) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:01:52.222358 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_db.sqlalchemy.engines [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103283) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:01:52.225771 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG dbcounter [-] [103284] Writer thread running {{(pid=103284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:01:52.228748 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writer thread running {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:01:52.281767 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.context [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),21c70492-35f8-4468-a997-987c705bd844(cell1) {{(pid=103284) load_cells /opt/stack/nova/nova/context.py:464}} Jul 05 10:01:52.284218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.context [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),21c70492-35f8-4468-a997-987c705bd844(cell1) {{(pid=103283) load_cells /opt/stack/nova/nova/context.py:464}} Jul 05 10:01:52.286035 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG oslo_concurrency.lockutils [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103284) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:01:52.287691 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG oslo_concurrency.lockutils [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103284) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:01:52.288170 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:01:52.288498 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG oslo_concurrency.lockutils [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103284) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:01:52.289771 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:01:52.290576 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:01:52.290683 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG oslo_concurrency.lockutils [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103284) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:01:52.292633 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO dbcounter [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] Registered counter for database nova_cell0 Jul 05 10:01:52.292729 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:01:52.294606 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO dbcounter [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Registered counter for database nova_cell0 Jul 05 10:01:52.296592 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO dbcounter [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] Registered counter for database nova_cell1 Jul 05 10:01:52.298411 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO dbcounter [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Registered counter for database nova_cell1 Jul 05 10:01:52.304795 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG oslo_db.sqlalchemy.engines [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103284) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:01:52.306452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG oslo_db.sqlalchemy.engines [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103284) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:01:52.307905 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_db.sqlalchemy.engines [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103283) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:01:52.311662 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_db.sqlalchemy.engines [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103283) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:01:52.311801 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG dbcounter [-] [103284] Writer thread running {{(pid=103284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:01:52.314144 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writer thread running {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:01:52.449894 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG dbcounter [-] [103284] Writer thread running {{(pid=103284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:01:52.451745 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writer thread running {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:01:52.460021 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO dbcounter [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] Registered counter for database nova_cell0 Jul 05 10:01:52.461046 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO dbcounter [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Registered counter for database nova_cell0 Jul 05 10:01:52.468680 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG oslo_db.sqlalchemy.engines [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103284) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:01:52.472120 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_db.sqlalchemy.engines [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103283) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:01:52.473886 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG dbcounter [-] [103284] Writer thread running {{(pid=103284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:01:52.476945 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writer thread running {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:01:52.805907 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: WARNING keystonemiddleware.auth_token [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 05 10:01:52.810272 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: WARNING keystonemiddleware.auth_token [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 05 10:01:52.813055 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: WARNING keystonemiddleware.auth_token [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 05 10:01:52.818011 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: WARNING keystonemiddleware.auth_token [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 05 10:01:53.149233 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: WARNING keystonemiddleware.auth_token [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 05 10:01:53.155039 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: WARNING keystonemiddleware.auth_token [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 05 10:01:53.157677 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: WARNING keystonemiddleware.auth_token [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 05 10:01:53.161029 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f5054459c0 pid: 103283 (default app) Jul 05 10:01:53.163074 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:01:53.164114 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: WARNING keystonemiddleware.auth_token [None req-55e57162-5641-49da-a04e-0542aee0fe40 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 05 10:01:53.164246 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] 10.222.0.20 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001721 Jul 05 10:01:53.164685 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 788 bytes} [Tue Jul 5 10:01:53 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:01:53.169415 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f5054459c0 pid: 103284 (default app) Jul 05 10:03:06.788473 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:06.789476 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002236 Jul 05 10:03:06.789864 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 2/2] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:03:06 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:06.801912 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG dbcounter [-] [103284] Writing DB stats nova_api:SELECT=3 {{(pid=103284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:06.803147 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG dbcounter [-] [103284] Writing DB stats nova_cell0:SELECT=3 {{(pid=103284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:06.804817 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG dbcounter [-] [103284] Writing DB stats nova_cell1:SELECT=3 {{(pid=103284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:06.805694 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG dbcounter [-] [103284] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:07.242655 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-6573d0b6-3493-4611-9667-3b1628ded4d6 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:07.273849 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-6573d0b6-3493-4611-9667-3b1628ded4d6 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.477399 Jul 05 10:03:07.274745 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 1/3] 10.222.0.20 () {60 vars in 1291 bytes} [Tue Jul 5 10:03:06 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:03:09.335657 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:09.336778 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-70df3216-f0d9-44eb-ba93-4f007d8634b2 None None] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002071 Jul 05 10:03:09.337343 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 3/4] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:03:09 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:09.397775 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-fba349e0-7b38-477e-bbef-385e40ae6415 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "vcpus": 1, "disk": 1, "name": "cirros256", "id": "c1", "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "ram": 256}} {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:09.418992 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:03:09.418992 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: warnings.warn(msg) Jul 05 10:03:09.446875 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-fba349e0-7b38-477e-bbef-385e40ae6415 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.103484 Jul 05 10:03:09.447571 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 2/5] 10.222.0.20 () {64 vars in 1282 bytes} [Tue Jul 5 10:03:09 2022] POST /compute/v2.1/flavors => generated 407 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 05 10:03:09.462896 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writing DB stats nova_api:SELECT=3 {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:09.464263 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writing DB stats nova_cell0:SELECT=3 {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:09.465623 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writing DB stats nova_cell1:SELECT=3 {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:09.466464 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:09.670486 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-49e0559d-318d-4e3c-a093-7d75f2267112 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:09.672831 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:03:09.672831 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: warnings.warn(msg) Jul 05 10:03:09.732869 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-49e0559d-318d-4e3c-a093-7d75f2267112 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.276612 Jul 05 10:03:09.739380 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 4/6] 10.222.0.20 () {64 vars in 1335 bytes} [Tue Jul 5 10:03:09 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:03:11.810378 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-fba349e0-7b38-477e-bbef-385e40ae6415 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:11.812056 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-fba349e0-7b38-477e-bbef-385e40ae6415 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002241 Jul 05 10:03:11.813045 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 3/7] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:03:11 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:11.919135 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-5a75e8a5-97cf-462f-926f-365cc40de3f9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 5, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "name": "ds512M", "vcpus": 1, "ram": 512, "id": "d1", "os-flavor-access:is_public": true}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:11.937764 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:03:11.937764 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: warnings.warn(msg) Jul 05 10:03:11.959097 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-5a75e8a5-97cf-462f-926f-365cc40de3f9 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.140212 Jul 05 10:03:11.959656 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 5/8] 10.222.0.20 () {64 vars in 1282 bytes} [Tue Jul 5 10:03:11 2022] POST /compute/v2.1/flavors => generated 404 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:03:11.971502 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-15bb4b5a-f6e0-4b91-acf7-0be1dc7b3bcd admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:11.973169 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:03:11.973169 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: warnings.warn(msg) Jul 05 10:03:12.005409 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-15bb4b5a-f6e0-4b91-acf7-0be1dc7b3bcd admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.037498 Jul 05 10:03:12.005910 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 4/9] 10.222.0.20 () {64 vars in 1335 bytes} [Tue Jul 5 10:03:11 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:03:14.065576 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-5a75e8a5-97cf-462f-926f-365cc40de3f9 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:14.066586 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-5a75e8a5-97cf-462f-926f-365cc40de3f9 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001453 Jul 05 10:03:14.067388 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 6/10] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:03:14 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:14.131633 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-41dce91d-c802-42cc-b0fc-e309972c66ab admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "disk": 10, "swap": 0, "name": "ds1G", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "ram": 1024, "id": "d2", "os-flavor-access:is_public": true}} {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:14.150824 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-41dce91d-c802-42cc-b0fc-e309972c66ab admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.077236 Jul 05 10:03:14.151474 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 5/11] 10.222.0.20 () {64 vars in 1282 bytes} [Tue Jul 5 10:03:14 2022] POST /compute/v2.1/flavors => generated 404 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 05 10:03:14.164317 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-d3f4eb87-bc18-477b-ab8d-90d86c631274 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:14.187344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-d3f4eb87-bc18-477b-ab8d-90d86c631274 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027627 Jul 05 10:03:14.187834 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 7/12] 10.222.0.20 () {64 vars in 1335 bytes} [Tue Jul 5 10:03:14 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:03:16.257951 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-41dce91d-c802-42cc-b0fc-e309972c66ab admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:16.258959 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-41dce91d-c802-42cc-b0fc-e309972c66ab admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001447 Jul 05 10:03:16.259498 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 6/13] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:03:16 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:16.317965 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-efd5fc31-c2a9-4a37-8aff-3cb2a71c0303 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d3", "ram": 2048, "vcpus": 2, "rxtx_factor": 1.0, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds2G", "disk": 10, "os-flavor-access:is_public": true}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:16.335843 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-efd5fc31-c2a9-4a37-8aff-3cb2a71c0303 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.070371 Jul 05 10:03:16.336385 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 8/14] 10.222.0.20 () {64 vars in 1282 bytes} [Tue Jul 5 10:03:16 2022] POST /compute/v2.1/flavors => generated 404 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:03:16.347937 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-4ffe924b-952d-4217-99bc-1a62b8cf75e4 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:16.370181 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-4ffe924b-952d-4217-99bc-1a62b8cf75e4 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.025858 Jul 05 10:03:16.371030 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 7/15] 10.222.0.20 () {64 vars in 1335 bytes} [Tue Jul 5 10:03:16 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:03:18.452124 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-efd5fc31-c2a9-4a37-8aff-3cb2a71c0303 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:18.453164 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-efd5fc31-c2a9-4a37-8aff-3cb2a71c0303 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001477 Jul 05 10:03:18.453552 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 9/16] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:03:18 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:18.514564 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-0a3031f3-e01b-4aae-bfe8-8e152597585b admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 4096, "id": "d4", "name": "ds4G", "disk": 20, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "vcpus": 4, "os-flavor-access:is_public": true}} {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:18.532054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-0a3031f3-e01b-4aae-bfe8-8e152597585b admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.072728 Jul 05 10:03:18.532644 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 8/17] 10.222.0.20 () {64 vars in 1282 bytes} [Tue Jul 5 10:03:18 2022] POST /compute/v2.1/flavors => generated 404 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:03:18.545144 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-08eda96e-a215-4fde-be05-4541109a3228 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:18.566451 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-08eda96e-a215-4fde-be05-4541109a3228 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.024936 Jul 05 10:03:18.566997 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 10/18] 10.222.0.20 () {64 vars in 1335 bytes} [Tue Jul 5 10:03:18 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:03:20.617421 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-0a3031f3-e01b-4aae-bfe8-8e152597585b admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:20.618428 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-0a3031f3-e01b-4aae-bfe8-8e152597585b admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001437 Jul 05 10:03:20.618822 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 9/19] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:03:20 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:20.678884 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-a4cec98b-8780-43c0-8c24-285c2f0341a2 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:20.703462 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-a4cec98b-8780-43c0-8c24-285c2f0341a2 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.078741 Jul 05 10:03:20.704225 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 11/20] 10.222.0.20 () {60 vars in 1291 bytes} [Tue Jul 5 10:03:20 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 05 10:03:22.822511 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-0a3031f3-e01b-4aae-bfe8-8e152597585b admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:22.823526 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-0a3031f3-e01b-4aae-bfe8-8e152597585b admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001447 Jul 05 10:03:22.823937 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 10/21] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:03:22 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:22.884222 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-f6b92377-cf63-4d32-9f6d-1dc19c156bba admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "id": "1", "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 1, "swap": 0, "ram": 512, "os-flavor-access:is_public": true, "vcpus": 1, "name": "m1.tiny"}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:22.903078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-f6b92377-cf63-4d32-9f6d-1dc19c156bba admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.072752 Jul 05 10:03:22.903676 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 12/22] 10.222.0.20 () {64 vars in 1282 bytes} [Tue Jul 5 10:03:22 2022] POST /compute/v2.1/flavors => generated 402 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 05 10:03:22.919277 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-4e2fc8c2-a185-4f0d-b128-f56e026e21ca admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:22.951717 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-4e2fc8c2-a185-4f0d-b128-f56e026e21ca admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.039096 Jul 05 10:03:22.952465 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 11/23] 10.222.0.20 () {64 vars in 1332 bytes} [Tue Jul 5 10:03:22 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:03:25.175613 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-f6b92377-cf63-4d32-9f6d-1dc19c156bba admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:25.176580 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-f6b92377-cf63-4d32-9f6d-1dc19c156bba admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001456 Jul 05 10:03:25.176995 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 13/24] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:03:25 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:25.237329 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-7c98e5d7-1999-4099-a560-ad841e57224f admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "2", "swap": 0, "os-flavor-access:is_public": true, "name": "m1.small", "rxtx_factor": 1.0, "ram": 2048, "disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1}} {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:25.258410 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-7c98e5d7-1999-4099-a560-ad841e57224f admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.074119 Jul 05 10:03:25.259131 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 12/25] 10.222.0.20 () {64 vars in 1282 bytes} [Tue Jul 5 10:03:25 2022] POST /compute/v2.1/flavors => generated 405 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:03:25.270839 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-a1d21802-1076-4c70-985a-fd9c2708a7f2 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:25.293745 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-a1d21802-1076-4c70-985a-fd9c2708a7f2 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.026527 Jul 05 10:03:25.294324 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 14/26] 10.222.0.20 () {64 vars in 1332 bytes} [Tue Jul 5 10:03:25 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:03:27.379365 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-7c98e5d7-1999-4099-a560-ad841e57224f admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:27.380377 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-7c98e5d7-1999-4099-a560-ad841e57224f admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001460 Jul 05 10:03:27.380809 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 13/27] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:03:27 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:27.466828 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-9a0024bb-44eb-4f0f-a40e-3c6cd078654b admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "3", "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 2, "name": "m1.medium", "swap": 0, "disk": 40, "rxtx_factor": 1.0, "ram": 4096}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:27.485525 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-9a0024bb-44eb-4f0f-a40e-3c6cd078654b admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.098334 Jul 05 10:03:27.486067 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 15/28] 10.222.0.20 () {64 vars in 1282 bytes} [Tue Jul 5 10:03:27 2022] POST /compute/v2.1/flavors => generated 406 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:03:27.498455 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-65fea3f5-f912-4834-b811-975ebe015ec3 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:27.521921 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-65fea3f5-f912-4834-b811-975ebe015ec3 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027106 Jul 05 10:03:27.522490 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 14/29] 10.222.0.20 () {64 vars in 1332 bytes} [Tue Jul 5 10:03:27 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:03:29.566274 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-9a0024bb-44eb-4f0f-a40e-3c6cd078654b admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:29.567273 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-9a0024bb-44eb-4f0f-a40e-3c6cd078654b admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001428 Jul 05 10:03:29.567671 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 16/30] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:03:29 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:29.624463 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-88ec83a5-4dd2-4ad6-8789-d4a168fbe58b admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 4, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.large", "ram": 8192, "rxtx_factor": 1.0, "disk": 80, "id": "4", "os-flavor-access:is_public": true, "swap": 0}} {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:29.641566 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-88ec83a5-4dd2-4ad6-8789-d4a168fbe58b admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.068293 Jul 05 10:03:29.642097 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 15/31] 10.222.0.20 () {64 vars in 1282 bytes} [Tue Jul 5 10:03:29 2022] POST /compute/v2.1/flavors => generated 405 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 05 10:03:29.654293 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-89cd9dd2-4855-48ff-be84-dcc9e2ac7911 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:29.676193 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-89cd9dd2-4855-48ff-be84-dcc9e2ac7911 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.025536 Jul 05 10:03:29.676752 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 17/32] 10.222.0.20 () {64 vars in 1332 bytes} [Tue Jul 5 10:03:29 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:03:31.719015 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-88ec83a5-4dd2-4ad6-8789-d4a168fbe58b admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:31.720032 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-88ec83a5-4dd2-4ad6-8789-d4a168fbe58b admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001452 Jul 05 10:03:31.720498 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 16/33] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:03:31 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:31.777924 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-f7e7dc38-9a6d-43b8-81e0-b1050ead4e7a admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 160, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 8, "ram": 16384, "os-flavor-access:is_public": true, "id": "5", "name": "m1.xlarge", "rxtx_factor": 1.0}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:31.806924 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-f7e7dc38-9a6d-43b8-81e0-b1050ead4e7a admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.080545 Jul 05 10:03:31.807468 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 18/34] 10.222.0.20 () {64 vars in 1282 bytes} [Tue Jul 5 10:03:31 2022] POST /compute/v2.1/flavors => generated 408 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:03:31.819365 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-b9dbab92-f379-4b97-938e-0cfa33336ace admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:03:31.845166 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-b9dbab92-f379-4b97-938e-0cfa33336ace admin admin] 10.222.0.20 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029531 Jul 05 10:03:31.845995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 17/35] 10.222.0.20 () {64 vars in 1332 bytes} [Tue Jul 5 10:03:31 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:03:55.999435 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-f7e7dc38-9a6d-43b8-81e0-b1050ead4e7a admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:56.000476 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-f7e7dc38-9a6d-43b8-81e0-b1050ead4e7a admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001733 Jul 05 10:03:56.001331 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 19/36] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:03:55 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:56.011970 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG dbcounter [-] [103284] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=10,nova_api:SAVEPOINT=5,nova_api:RELEASE=5 {{(pid=103284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:56.060301 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-ed631ef1-2327-4053-b9e5-2b3832630713 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:56.077399 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-ed631ef1-2327-4053-b9e5-2b3832630713 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.070065 Jul 05 10:03:56.077937 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 18/37] 10.222.0.20 () {60 vars in 1291 bytes} [Tue Jul 5 10:03:56 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:03:56.981346 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-f7e7dc38-9a6d-43b8-81e0-b1050ead4e7a admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:56.982341 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-f7e7dc38-9a6d-43b8-81e0-b1050ead4e7a admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001400 Jul 05 10:03:56.982734 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 20/38] 10.222.0.20 () {56 vars in 1074 bytes} [Tue Jul 5 10:03:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:57.171187 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-c6fc2364-4ce8-4a74-8413-86ddcd5f441e admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:57.176806 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.wsgi [None req-c6fc2364-4ce8-4a74-8413-86ddcd5f441e admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-c6fc2364-4ce8-4a74-8413-86ddcd5f441e admin admin] Request method failure captured: Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Accept: application/json Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Accept-Encoding: gzip, deflate Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Connection: keep-alive Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Content-Length: 0 Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Host: 10.222.0.20 Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Openstack-Api-Version: compute 2.90 Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Openstack-System-Scope: None Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: User-Agent: python-novaclient Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Auth-Token: gAAAAABixAyMLfpWUgB7Xe-nRHPvK6NE5dOPSwdP4gzMwCKHD2_vrfe9aiQ40Z3U5MpvTN6Nq_VvCIWoq1mTMw9ZVvdbkjU0SeZb13jAacZM_OQ7CvewRlDVROtRIFd01g66W5Eq9Fm_Cc7D92l45sMq1e4vL7cI8LMOwj98jAqrmLXMDmdHVW4 Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Domain-Id: None Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Domain-Name: None Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Identity-Status: Confirmed Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Is-Admin-Project: True Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Openstack-Nova-Api-Version: 2.90 Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Project-Domain-Id: default Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Project-Domain-Name: Default Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Project-Id: ba1e700aae1b46039e859b8f609bb931 Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Project-Name: admin Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Role: member,reader,admin Jul 05 10:03:57.180054 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Roles: member,reader,admin Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/ba1e700aae1b46039e859b8f609bb931"}]}] Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Tenant: admin Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Tenant-Id: ba1e700aae1b46039e859b8f609bb931 Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Tenant-Name: admin Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User: admin Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User-Domain-Id: default Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User-Domain-Name: Default Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User-Id: bb9174aa3ac54d708fcd3d9dc7b9b827 Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User-Name: admin Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: method: > Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: exception: Flavor 42 could not be found. Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: action_args: {'id': '42'} Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:03:57.182050 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:03:57.183196 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:57.184693 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-c6fc2364-4ce8-4a74-8413-86ddcd5f441e admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103284) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:03:57.185960 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-c6fc2364-4ce8-4a74-8413-86ddcd5f441e admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.063745 Jul 05 10:03:57.185960 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 19/39] 10.222.0.20 () {60 vars in 1190 bytes} [Tue Jul 5 10:03:57 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 05 10:03:57.191647 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-b6e01b96-02a3-4530-9a35-be28f4b992d4 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:57.194661 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:03:57.214190 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-b6e01b96-02a3-4530-9a35-be28f4b992d4 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.026127 Jul 05 10:03:57.214718 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 21/40] 10.222.0.20 () {60 vars in 1210 bytes} [Tue Jul 5 10:03:57 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:03:57.224233 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-b4396492-afc1-424c-94d8-8e74730c68a3 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:57.245480 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-b4396492-afc1-424c-94d8-8e74730c68a3 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.024757 Jul 05 10:03:57.245956 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 20/41] 10.222.0.20 () {60 vars in 1210 bytes} [Tue Jul 5 10:03:57 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 05 10:03:57.255168 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-77942559-a0cc-42be-adfd-cfd67f7cdb09 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:57.261941 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.wsgi [None req-77942559-a0cc-42be-adfd-cfd67f7cdb09 admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-77942559-a0cc-42be-adfd-cfd67f7cdb09 admin admin] Request method failure captured: Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Accept: application/json Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Accept-Encoding: gzip, deflate Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Connection: keep-alive Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Content-Length: 0 Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Host: 10.222.0.20 Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Openstack-Api-Version: compute 2.90 Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Openstack-System-Scope: None Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: User-Agent: python-novaclient Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Auth-Token: gAAAAABixAyMLfpWUgB7Xe-nRHPvK6NE5dOPSwdP4gzMwCKHD2_vrfe9aiQ40Z3U5MpvTN6Nq_VvCIWoq1mTMw9ZVvdbkjU0SeZb13jAacZM_OQ7CvewRlDVROtRIFd01g66W5Eq9Fm_Cc7D92l45sMq1e4vL7cI8LMOwj98jAqrmLXMDmdHVW4 Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Domain-Id: None Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Domain-Name: None Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Identity-Status: Confirmed Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Is-Admin-Project: True Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Openstack-Nova-Api-Version: 2.90 Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Project-Domain-Id: default Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Project-Domain-Name: Default Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Project-Id: ba1e700aae1b46039e859b8f609bb931 Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Project-Name: admin Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Role: member,reader,admin Jul 05 10:03:57.265214 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Roles: member,reader,admin Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/ba1e700aae1b46039e859b8f609bb931"}]}] Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Tenant: admin Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Tenant-Id: ba1e700aae1b46039e859b8f609bb931 Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Tenant-Name: admin Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User: admin Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User-Domain-Id: default Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User-Domain-Name: Default Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User-Id: bb9174aa3ac54d708fcd3d9dc7b9b827 Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User-Name: admin Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: method: > Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: exception: Flavor 42 could not be found. Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: action_args: {'id': '42'} Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:03:57.266715 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:03:57.267645 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:57.268942 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-77942559-a0cc-42be-adfd-cfd67f7cdb09 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103283) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:03:57.270178 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-77942559-a0cc-42be-adfd-cfd67f7cdb09 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.014990 Jul 05 10:03:57.270178 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 22/42] 10.222.0.20 () {60 vars in 1190 bytes} [Tue Jul 5 10:03:57 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 05 10:03:58.069369 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-b4396492-afc1-424c-94d8-8e74730c68a3 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:58.070357 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-b4396492-afc1-424c-94d8-8e74730c68a3 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001405 Jul 05 10:03:58.070752 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 21/43] 10.222.0.20 () {56 vars in 1074 bytes} [Tue Jul 5 10:03:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:03:58.258320 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-827fcfed-d8cd-435f-8608-409bb4796c58 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:58.264060 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.wsgi [None req-827fcfed-d8cd-435f-8608-409bb4796c58 admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-827fcfed-d8cd-435f-8608-409bb4796c58 admin admin] Request method failure captured: Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Accept: application/json Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Accept-Encoding: gzip, deflate Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Connection: keep-alive Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Content-Length: 0 Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Host: 10.222.0.20 Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Openstack-Api-Version: compute 2.90 Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Openstack-System-Scope: None Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: User-Agent: python-novaclient Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Auth-Token: gAAAAABixAyOmrMoTncGlE6alOCdFS7oKvETDTplF2jBYOp9lCxPXVKEUeYH4DOIPz16wLi6zVwwcOZanu7OOyGJkCjh87u4R5l96470KtezSC_2mQmr4rerZ6CnADLzKHjYJ8H-cpqMEiMdfMADCsKPV2VXS4EMT_sy9EkUG9jvcniW8FsCWXY Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Domain-Id: None Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Domain-Name: None Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Identity-Status: Confirmed Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Is-Admin-Project: True Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Openstack-Nova-Api-Version: 2.90 Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Project-Domain-Id: default Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Project-Domain-Name: Default Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Project-Id: ba1e700aae1b46039e859b8f609bb931 Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Project-Name: admin Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Role: member,reader,admin Jul 05 10:03:58.265607 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Roles: member,reader,admin Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/ba1e700aae1b46039e859b8f609bb931"}]}] Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Tenant: admin Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Tenant-Id: ba1e700aae1b46039e859b8f609bb931 Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Tenant-Name: admin Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User: admin Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User-Domain-Id: default Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User-Domain-Name: Default Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User-Id: bb9174aa3ac54d708fcd3d9dc7b9b827 Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User-Name: admin Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: method: > Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: exception: Flavor 84 could not be found. Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: action_args: {'id': '84'} Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:03:58.267096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:03:58.269078 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:58.270344 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-827fcfed-d8cd-435f-8608-409bb4796c58 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103283) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:03:58.271651 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-827fcfed-d8cd-435f-8608-409bb4796c58 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.060288 Jul 05 10:03:58.271651 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 23/44] 10.222.0.20 () {60 vars in 1190 bytes} [Tue Jul 5 10:03:58 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 05 10:03:58.277126 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-a0b1a950-5146-4576-be46-ca63098c8597 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:58.291599 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-a0b1a950-5146-4576-be46-ca63098c8597 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018372 Jul 05 10:03:58.292136 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 22/45] 10.222.0.20 () {60 vars in 1210 bytes} [Tue Jul 5 10:03:58 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 05 10:03:58.301290 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-7f5cfb4d-04cf-4e09-be3f-be64c1efde3d admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:58.314547 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-7f5cfb4d-04cf-4e09-be3f-be64c1efde3d admin admin] 10.222.0.20 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.016632 Jul 05 10:03:58.315051 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 24/46] 10.222.0.20 () {60 vars in 1210 bytes} [Tue Jul 5 10:03:58 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:03:58.324243 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-947f620d-8f22-4a6c-8e3d-e90cb3afadf9 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:58.329475 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.wsgi [None req-947f620d-8f22-4a6c-8e3d-e90cb3afadf9 admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-947f620d-8f22-4a6c-8e3d-e90cb3afadf9 admin admin] Request method failure captured: Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Accept: application/json Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Accept-Encoding: gzip, deflate Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Connection: keep-alive Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Content-Length: 0 Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Host: 10.222.0.20 Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Openstack-Api-Version: compute 2.90 Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Openstack-System-Scope: None Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: User-Agent: python-novaclient Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Auth-Token: gAAAAABixAyOmrMoTncGlE6alOCdFS7oKvETDTplF2jBYOp9lCxPXVKEUeYH4DOIPz16wLi6zVwwcOZanu7OOyGJkCjh87u4R5l96470KtezSC_2mQmr4rerZ6CnADLzKHjYJ8H-cpqMEiMdfMADCsKPV2VXS4EMT_sy9EkUG9jvcniW8FsCWXY Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Domain-Id: None Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Domain-Name: None Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Identity-Status: Confirmed Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Is-Admin-Project: True Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Openstack-Nova-Api-Version: 2.90 Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Project-Domain-Id: default Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Project-Domain-Name: Default Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Project-Id: ba1e700aae1b46039e859b8f609bb931 Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Project-Name: admin Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Role: member,reader,admin Jul 05 10:03:58.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Roles: member,reader,admin Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/ba1e700aae1b46039e859b8f609bb931"}]}] Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Tenant: admin Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Tenant-Id: ba1e700aae1b46039e859b8f609bb931 Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Tenant-Name: admin Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User: admin Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User-Domain-Id: default Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User-Domain-Name: Default Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User-Id: bb9174aa3ac54d708fcd3d9dc7b9b827 Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User-Name: admin Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: method: > Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: exception: Flavor 84 could not be found. Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: action_args: {'id': '84'} Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:03:58.332532 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:03:58.333452 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:58.334763 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-947f620d-8f22-4a6c-8e3d-e90cb3afadf9 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103284) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:03:58.336030 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-947f620d-8f22-4a6c-8e3d-e90cb3afadf9 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.011992 Jul 05 10:03:58.336030 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 23/47] 10.222.0.20 () {60 vars in 1190 bytes} [Tue Jul 5 10:03:58 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Jul 05 10:03:59.155813 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-7f5cfb4d-04cf-4e09-be3f-be64c1efde3d admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:59.156836 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-7f5cfb4d-04cf-4e09-be3f-be64c1efde3d admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001427 Jul 05 10:03:59.157225 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 25/48] 10.222.0.20 () {56 vars in 1074 bytes} [Tue Jul 5 10:03:59 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:03:59.343415 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-f9f79c46-3474-4e53-ba1e-250ee949b216 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:59.349205 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.wsgi [None req-f9f79c46-3474-4e53-ba1e-250ee949b216 admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-f9f79c46-3474-4e53-ba1e-250ee949b216 admin admin] Request method failure captured: Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Accept: application/json Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Accept-Encoding: gzip, deflate Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Connection: keep-alive Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Content-Length: 0 Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Host: 10.222.0.20 Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Openstack-Api-Version: compute 2.90 Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Openstack-System-Scope: None Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: User-Agent: python-novaclient Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Auth-Token: gAAAAABixAyPGSjui9VlJh0PJGKLLQFx4Crf67fMvQXzi-hgeorMas-FpHjU3Vv4WJq8odZbreglXKgaSfgU7ksR-la7Tb5tib2x7zVdkiUSdlWWF2t7_lVWEMTDCMh4ynn3qY603yOIle_BHygCihWoAwD5yWuP4SAF7rnsiUrBjwHKHdLZYec Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Domain-Id: None Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Domain-Name: None Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Identity-Status: Confirmed Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Is-Admin-Project: True Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Openstack-Nova-Api-Version: 2.90 Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Project-Domain-Id: default Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Project-Domain-Name: Default Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Project-Id: ba1e700aae1b46039e859b8f609bb931 Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Project-Name: admin Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Role: member,reader,admin Jul 05 10:03:59.350703 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Roles: member,reader,admin Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/ba1e700aae1b46039e859b8f609bb931"}]}] Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Tenant: admin Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Tenant-Id: ba1e700aae1b46039e859b8f609bb931 Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-Tenant-Name: admin Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User: admin Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User-Domain-Id: default Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User-Domain-Name: Default Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User-Id: bb9174aa3ac54d708fcd3d9dc7b9b827 Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: X-User-Name: admin Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: method: > Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: exception: Flavor 451 could not be found. Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: action_args: {'id': '451'} Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:03:59.351998 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:03:59.353350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: ERROR nova.api.openstack.wsgi Jul 05 10:03:59.354583 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-f9f79c46-3474-4e53-ba1e-250ee949b216 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103284) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:03:59.355766 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-f9f79c46-3474-4e53-ba1e-250ee949b216 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.058730 Jul 05 10:03:59.355766 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 24/49] 10.222.0.20 () {60 vars in 1193 bytes} [Tue Jul 5 10:03:59 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 05 10:03:59.362130 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-7d60171d-bfb9-4f42-bde4-ca2317c7b031 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:59.375998 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-7d60171d-bfb9-4f42-bde4-ca2317c7b031 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017434 Jul 05 10:03:59.376582 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 26/50] 10.222.0.20 () {60 vars in 1210 bytes} [Tue Jul 5 10:03:59 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:03:59.385740 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:59.400350 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018059 Jul 05 10:03:59.400895 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 25/51] 10.222.0.20 () {60 vars in 1210 bytes} [Tue Jul 5 10:03:59 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 05 10:03:59.409716 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-9ac72254-f2f4-4f8d-bdf9-09a8b9b693d2 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:03:59.414873 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.wsgi [None req-9ac72254-f2f4-4f8d-bdf9-09a8b9b693d2 admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-9ac72254-f2f4-4f8d-bdf9-09a8b9b693d2 admin admin] Request method failure captured: Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Accept: application/json Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Accept-Encoding: gzip, deflate Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Connection: keep-alive Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Content-Length: 0 Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Host: 10.222.0.20 Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Openstack-Api-Version: compute 2.90 Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Openstack-System-Scope: None Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: User-Agent: python-novaclient Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Auth-Token: gAAAAABixAyPGSjui9VlJh0PJGKLLQFx4Crf67fMvQXzi-hgeorMas-FpHjU3Vv4WJq8odZbreglXKgaSfgU7ksR-la7Tb5tib2x7zVdkiUSdlWWF2t7_lVWEMTDCMh4ynn3qY603yOIle_BHygCihWoAwD5yWuP4SAF7rnsiUrBjwHKHdLZYec Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Domain-Id: None Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Domain-Name: None Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Identity-Status: Confirmed Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Is-Admin-Project: True Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Openstack-Nova-Api-Version: 2.90 Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Project-Domain-Id: default Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Project-Domain-Name: Default Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Project-Id: ba1e700aae1b46039e859b8f609bb931 Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Project-Name: admin Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Role: member,reader,admin Jul 05 10:03:59.417186 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Roles: member,reader,admin Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/ba1e700aae1b46039e859b8f609bb931"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/ba1e700aae1b46039e859b8f609bb931"}]}] Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Tenant: admin Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Tenant-Id: ba1e700aae1b46039e859b8f609bb931 Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-Tenant-Name: admin Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User: admin Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User-Domain-Id: default Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User-Domain-Name: Default Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User-Id: bb9174aa3ac54d708fcd3d9dc7b9b827 Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: X-User-Name: admin Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: method: > Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: exception: Flavor 451 could not be found. Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: action_args: {'id': '451'} Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:03:59.418770 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:03:59.419699 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: ERROR nova.api.openstack.wsgi Jul 05 10:03:59.421218 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-9ac72254-f2f4-4f8d-bdf9-09a8b9b693d2 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103283) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:03:59.422559 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-9ac72254-f2f4-4f8d-bdf9-09a8b9b693d2 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.012219 Jul 05 10:03:59.422559 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 27/52] 10.222.0.20 () {60 vars in 1193 bytes} [Tue Jul 5 10:03:59 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 05 10:04:00.229219 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:04:00.230250 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001510 Jul 05 10:04:00.230672 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 26/53] 10.222.0.20 () {56 vars in 1074 bytes} [Tue Jul 5 10:04:00 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:04:00.418681 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-7828b3b1-72d3-4927-bb25-17a9ea7abd97 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:04:00.437515 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-7828b3b1-72d3-4927-bb25-17a9ea7abd97 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.070884 Jul 05 10:04:00.438051 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 28/54] 10.222.0.20 () {64 vars in 1235 bytes} [Tue Jul 5 10:04:00 2022] POST /compute/v2.1/flavors => generated 404 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:04:01.250026 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:04:01.251022 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001407 Jul 05 10:04:01.251437 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 27/55] 10.222.0.20 () {56 vars in 1074 bytes} [Tue Jul 5 10:04:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:04:01.440905 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-9a18a95e-cb4f-4ea1-9026-4e422a733698 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:04:01.457507 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-9a18a95e-cb4f-4ea1-9026-4e422a733698 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.068143 Jul 05 10:04:01.458045 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 29/56] 10.222.0.20 () {64 vars in 1235 bytes} [Tue Jul 5 10:04:01 2022] POST /compute/v2.1/flavors => generated 405 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:04:02.265690 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:04:02.266721 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001437 Jul 05 10:04:02.267112 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 28/57] 10.222.0.20 () {56 vars in 1074 bytes} [Tue Jul 5 10:04:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:04:02.453309 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-5f801787-a80b-424c-a2e0-dbce0fc157ce admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:04:02.470096 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-5f801787-a80b-424c-a2e0-dbce0fc157ce admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.067955 Jul 05 10:04:02.470628 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 30/58] 10.222.0.20 () {64 vars in 1235 bytes} [Tue Jul 5 10:04:02 2022] POST /compute/v2.1/flavors => generated 407 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:04:03.278539 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:04:03.279536 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001413 Jul 05 10:04:03.279932 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 29/59] 10.222.0.20 () {56 vars in 1074 bytes} [Tue Jul 5 10:04:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:04:03.468151 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-1badf18c-2fcd-4511-8dc7-66a07dae8505 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:04:03.486515 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-1badf18c-2fcd-4511-8dc7-66a07dae8505 admin admin] 10.222.0.20 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.069705 Jul 05 10:04:03.487052 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 31/60] 10.222.0.20 () {64 vars in 1235 bytes} [Tue Jul 5 10:04:03 2022] POST /compute/v2.1/flavors => generated 409 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 05 10:04:04.316106 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:04:04.317228 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001491 Jul 05 10:04:04.317609 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 30/61] 10.222.0.20 () {56 vars in 1074 bytes} [Tue Jul 5 10:04:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:04:04.515057 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-1361f1dd-9e95-420a-9eff-6d9cc142de59 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:04:04.533842 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-1361f1dd-9e95-420a-9eff-6d9cc142de59 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.069527 Jul 05 10:04:04.534299 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 32/62] 10.222.0.20 () {60 vars in 1202 bytes} [Tue Jul 5 10:04:04 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:04:51.455417 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:04:51.456424 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-e065c8dd-4ad8-46b3-b0ec-9d0a2a8520c8 admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001405 Jul 05 10:04:51.456862 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 31/63] 10.222.0.20 () {54 vars in 930 bytes} [Tue Jul 5 10:04:51 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:04:51.467143 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:51.514969 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-cd4cb7fb-cfbd-4196-9b00-bff89df7959f admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:04:51.533573 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-cd4cb7fb-cfbd-4196-9b00-bff89df7959f admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.070836 Jul 05 10:04:51.534098 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 33/64] 10.222.0.20 () {60 vars in 1291 bytes} [Tue Jul 5 10:04:51 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:04:51.557119 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-8d5cdf77-14e7-4769-9a6b-bfd0175781d7 admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:04:51.561239 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG dbcounter [-] [103284] Writing DB stats nova_api:SELECT=14 {{(pid=103284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:04:51.566784 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-8d5cdf77-14e7-4769-9a6b-bfd0175781d7 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013256 Jul 05 10:04:51.567297 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 32/65] 10.222.0.20 () {60 vars in 1282 bytes} [Tue Jul 5 10:04:51 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:04:51.578335 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-72bafb04-b833-49b3-a14b-cabc30f18425 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:04:51.585930 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-72bafb04-b833-49b3-a14b-cabc30f18425 admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011028 Jul 05 10:04:51.586440 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 34/66] 10.222.0.20 () {60 vars in 1285 bytes} [Tue Jul 5 10:04:51 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:04:51.597316 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-1d13c8d2-881f-4a84-a831-306c5a44c93a admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:04:51.604609 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-1d13c8d2-881f-4a84-a831-306c5a44c93a admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010671 Jul 05 10:04:51.605110 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 33/67] 10.222.0.20 () {60 vars in 1285 bytes} [Tue Jul 5 10:04:51 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:04:51.616056 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-7f1717d5-65cf-4a59-b85a-91848d02f13c admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:04:51.623931 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-7f1717d5-65cf-4a59-b85a-91848d02f13c admin admin] 10.222.0.20 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011152 Jul 05 10:04:51.624448 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 35/68] 10.222.0.20 () {60 vars in 1282 bytes} [Tue Jul 5 10:04:51 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:08:04.002888 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-1d13c8d2-881f-4a84-a831-306c5a44c93a admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:08:04.003945 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-1d13c8d2-881f-4a84-a831-306c5a44c93a admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001508 Jul 05 10:08:04.004333 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 34/69] 10.222.0.20 () {56 vars in 1074 bytes} [Tue Jul 5 10:08:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:08:04.145021 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writing DB stats nova_api:SELECT=6 {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:08:04.256506 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-c0353a41-02fc-47d0-8cf4-4fe133c74258 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:08:04.257718 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:08:04.257718 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: warnings.warn(msg) Jul 05 10:08:04.259402 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-c0353a41-02fc-47d0-8cf4-4fe133c74258 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:08:04.259840 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-c0353a41-02fc-47d0-8cf4-4fe133c74258 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:08:04.260646 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-c0353a41-02fc-47d0-8cf4-4fe133c74258 admin admin] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:08:04.261076 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-c0353a41-02fc-47d0-8cf4-4fe133c74258 admin admin] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:08:04.294188 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-c0353a41-02fc-47d0-8cf4-4fe133c74258 admin admin] 10.222.0.20 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.150587 Jul 05 10:08:04.294731 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 36/70] 10.222.0.20 () {60 vars in 1193 bytes} [Tue Jul 5 10:08:04 2022] GET /compute/v2.1/os-services => generated 791 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:08:05.114841 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-1d13c8d2-881f-4a84-a831-306c5a44c93a admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:08:05.115875 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-1d13c8d2-881f-4a84-a831-306c5a44c93a admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001419 Jul 05 10:08:05.116263 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 35/71] 10.222.0.20 () {56 vars in 1074 bytes} [Tue Jul 5 10:08:05 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:08:05.305808 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-e3e329f6-544a-4901-a89d-54d944f92eb7 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:08:05.307011 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:08:05.307011 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: warnings.warn(msg) Jul 05 10:08:05.308181 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-e3e329f6-544a-4901-a89d-54d944f92eb7 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:08:05.308654 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-e3e329f6-544a-4901-a89d-54d944f92eb7 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:08:05.309439 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-e3e329f6-544a-4901-a89d-54d944f92eb7 admin admin] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:08:05.310133 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-e3e329f6-544a-4901-a89d-54d944f92eb7 admin admin] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:08:05.329164 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-e3e329f6-544a-4901-a89d-54d944f92eb7 admin admin] 10.222.0.20 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.074832 Jul 05 10:08:05.329695 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 37/72] 10.222.0.20 () {60 vars in 1193 bytes} [Tue Jul 5 10:08:05 2022] GET /compute/v2.1/os-services => generated 791 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:08:16.580808 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-1d13c8d2-881f-4a84-a831-306c5a44c93a admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:08:16.581812 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-1d13c8d2-881f-4a84-a831-306c5a44c93a admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001451 Jul 05 10:08:16.582202 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 36/73] 10.222.0.20 () {56 vars in 1074 bytes} [Tue Jul 5 10:08:16 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:08:16.724470 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writing DB stats nova_cell0:SELECT=4 {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:08:16.725191 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writing DB stats nova_cell1:SELECT=4 {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:08:16.725885 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG dbcounter [-] [103283] Writing DB stats nova_api:SELECT=8 {{(pid=103283) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:08:16.774216 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-1b55f7d1-9dc0-45f7-b752-6b37ea373a84 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:08:16.775307 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:08:16.775307 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: warnings.warn(msg) Jul 05 10:08:16.777050 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-1b55f7d1-9dc0-45f7-b752-6b37ea373a84 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:08:16.777483 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-1b55f7d1-9dc0-45f7-b752-6b37ea373a84 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:08:16.778255 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-1b55f7d1-9dc0-45f7-b752-6b37ea373a84 admin admin] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:08:16.778679 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-1b55f7d1-9dc0-45f7-b752-6b37ea373a84 admin admin] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:08:16.803517 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-1b55f7d1-9dc0-45f7-b752-6b37ea373a84 admin admin] 10.222.0.20 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.083564 Jul 05 10:08:16.804066 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 38/74] 10.222.0.20 () {60 vars in 1193 bytes} [Tue Jul 5 10:08:16 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:08:17.618705 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: DEBUG nova.api.openstack.wsgi [None req-1d13c8d2-881f-4a84-a831-306c5a44c93a admin admin] Calling method '>' {{(pid=103284) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:08:17.619710 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: INFO nova.api.openstack.requestlog [None req-1d13c8d2-881f-4a84-a831-306c5a44c93a admin admin] 10.222.0.20 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001422 Jul 05 10:08:17.620096 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: [pid: 103284|app: 0|req: 37/75] 10.222.0.20 () {56 vars in 1074 bytes} [Tue Jul 5 10:08:17 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:08:17.809411 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG nova.api.openstack.wsgi [None req-80bc6d01-e522-4e87-bcbd-bb893f91cc30 admin admin] Calling method '>' {{(pid=103283) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:08:17.810640 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 10:08:17.810640 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: warnings.warn(msg) Jul 05 10:08:17.812155 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-80bc6d01-e522-4e87-bcbd-bb893f91cc30 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:08:17.812638 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-80bc6d01-e522-4e87-bcbd-bb893f91cc30 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:08:17.813457 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-80bc6d01-e522-4e87-bcbd-bb893f91cc30 admin admin] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:08:17.813889 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: DEBUG oslo_concurrency.lockutils [None req-80bc6d01-e522-4e87-bcbd-bb893f91cc30 admin admin] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103283) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:08:17.835995 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: INFO nova.api.openstack.requestlog [None req-80bc6d01-e522-4e87-bcbd-bb893f91cc30 admin admin] 10.222.0.20 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.078138 Jul 05 10:08:17.836488 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: [pid: 103283|app: 0|req: 39/76] 10.222.0.20 () {60 vars in 1193 bytes} [Tue Jul 5 10:08:17 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:09:16.316783 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: Tue Jul 5 10:09:16 2022 - graceful shutdown triggered... Jul 05 10:09:16.316645 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Jul 05 10:09:16.317729 dv-cinder-master-3-iscsi devstack@n-api.service[103283]: Gracefully killing worker 1 (pid: 103283)... Jul 05 10:09:16.318125 dv-cinder-master-3-iscsi devstack@n-api.service[103284]: Gracefully killing worker 2 (pid: 103284)... Jul 05 10:09:17.317665 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: worker 1 buried after 1 seconds Jul 05 10:09:17.318215 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: worker 2 buried after 1 seconds Jul 05 10:09:17.318215 dv-cinder-master-3-iscsi devstack@n-api.service[103282]: goodbye to uWSGI. Jul 05 10:09:17.319301 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Jul 05 10:09:17.319847 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Jul 05 10:09:17.322728 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jul 05 10:09:17.332712 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 05 10:09:17.334841 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 5 10:09:17 2022] *** Jul 05 10:09:17.334841 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 05 10:09:17.334841 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 05 10:09:17.334841 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: nodename: dv-cinder-master-3-iscsi Jul 05 10:09:17.334841 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: machine: x86_64 Jul 05 10:09:17.334841 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: clock source: unix Jul 05 10:09:17.334841 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: pcre jit disabled Jul 05 10:09:17.334841 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: detected number of CPU cores: 4 Jul 05 10:09:17.334841 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: current working directory: / Jul 05 10:09:17.334841 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: detected binary path: /usr/bin/uwsgi-core Jul 05 10:09:17.335522 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: your processes number limit is 47680 Jul 05 10:09:17.335522 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: your memory page size is 4096 bytes Jul 05 10:09:17.335522 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: detected max file descriptor number: 2048 Jul 05 10:09:17.335522 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: lock engine: pthread robust mutexes Jul 05 10:09:17.335522 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: thunder lock: enabled Jul 05 10:09:17.335522 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 05 10:09:17.335522 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 05 10:09:17.381332 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: Python main interpreter initialized at 0x5581072f69c0 Jul 05 10:09:17.381332 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: python threads support enabled Jul 05 10:09:17.381332 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: your server socket listen backlog is limited to 100 connections Jul 05 10:09:17.381332 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: your mercy for graceful operations on workers is 90 seconds Jul 05 10:09:17.381804 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: mapped 403077 bytes (393 KB) for 2 cores Jul 05 10:09:17.381804 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: *** Operational MODE: preforking *** Jul 05 10:09:17.382041 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: *** uWSGI is running in multiple interpreter mode *** Jul 05 10:09:17.382041 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: spawned uWSGI master process (pid: 117793) Jul 05 10:09:17.382253 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jul 05 10:09:17.383817 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: spawned uWSGI worker 1 (pid: 117812, cores: 1) Jul 05 10:09:17.383817 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: spawned uWSGI worker 2 (pid: 117813, cores: 1) Jul 05 10:09:17.383817 dv-cinder-master-3-iscsi devstack@n-api.service[117793]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 05 10:09:18.032757 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 05 10:09:18.036004 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 05 10:09:19.222632 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 10:09:19.222632 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: from cryptography.utils import int_from_bytes Jul 05 10:09:19.223290 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 10:09:19.223290 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: from cryptography.utils import int_from_bytes Jul 05 10:09:19.234599 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 10:09:19.234599 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: from cryptography.utils import int_from_bytes Jul 05 10:09:19.235063 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 10:09:19.235063 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: from cryptography.utils import int_from_bytes Jul 05 10:09:21.500601 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 05 10:09:21.501142 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 05 10:09:21.501436 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 05 10:09:21.501745 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 05 10:09:21.502005 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 05 10:09:21.502418 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.502753 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.503077 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.503399 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.503733 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.504047 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.504370 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.504724 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.505055 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.505319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.505635 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.505963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.506286 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.506599 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.506911 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.507231 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.507539 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.507869 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.508167 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.508702 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.510286 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.510612 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.510939 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.511285 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.511610 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.511927 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.512252 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.512598 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.512922 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.513236 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.513655 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.513990 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.514294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.514614 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.515034 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.515350 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.515657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.515981 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.516305 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.516644 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.516976 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.517298 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.517620 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.517943 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.518794 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.519106 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.519415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.519738 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.520047 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.520354 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.520633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.520942 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.521265 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.521589 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.521933 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.522167 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.522418 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.522740 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.523061 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.523370 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.523678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.523986 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.524296 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.524627 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.524938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.525242 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.525546 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.525864 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.526180 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.526500 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.526805 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.527125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.527525 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.20 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.527774 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.20 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.528081 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.528455 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.528777 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.529090 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.529436 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.529729 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.530035 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.530340 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.530654 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.530822 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 05 10:09:21.531306 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.531470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 05 10:09:21.531621 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.531762 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 05 10:09:21.531916 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.531916 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.532190 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 05 10:09:21.532190 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 05 10:09:21.532499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.532499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.532817 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.532961 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.533108 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.533259 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.533411 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.533551 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.534362 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.534526 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.534669 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.534818 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.535011 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.535167 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.535309 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.535458 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.535598 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.535749 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.535909 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.536059 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.536207 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.536360 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.536561 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.536704 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.536855 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.537006 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.537147 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.537286 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.537425 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.537570 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.537724 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.537884 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.538034 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.538182 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.538330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.538470 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.539015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.539194 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.539360 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.539522 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.539522 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.539828 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.540326 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.540506 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.540663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.540817 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.540959 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.541123 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.541396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.541552 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.541695 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.541845 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.541986 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.542135 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.542284 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.542436 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.542584 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.542733 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.542879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.543025 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.543166 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.543314 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.543469 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.543649 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.543794 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.543944 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.544092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.544326 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.544502 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.544645 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.544785 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.544785 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.545219 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.545372 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.545520 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.545667 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.545825 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.545979 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.546131 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.546282 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.546422 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.546568 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.546709 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.547391 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.547502 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.547580 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.547662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.547742 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.547822 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.547908 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.547981 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.548058 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.548058 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.548208 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.548300 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.548438 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.548539 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.548623 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.548701 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.548778 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.548865 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.548865 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.549001 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.549080 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.549152 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.549232 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.549338 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.549417 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.549495 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.549820 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.549922 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.550014 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.550097 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.550199 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.550283 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.550478 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.550583 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.550756 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.550859 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.551045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.551153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.551349 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.551458 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.551655 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.551762 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.551959 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.552072 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.552262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.552470 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.552623 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.552775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.552926 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.553077 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.553228 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.553386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.553542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.553687 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.553830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.553986 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.554135 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.554287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.554449 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.554607 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.554754 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.554898 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.555052 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.20 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.555206 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.555351 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.20 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.555502 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.555646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.555791 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.555949 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.556094 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.556237 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.556418 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.556571 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.556726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.556881 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.557026 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.557169 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.557319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.557464 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.557613 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.557770 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.557929 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.558073 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.558225 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.558368 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.558518 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.558674 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.558825 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.558970 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.559120 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.559263 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.559419 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.559574 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.559729 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.559879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.560024 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.560444 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.560613 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.560760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.560904 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.561049 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.561199 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.561354 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.561514 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.561659 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.561811 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.561955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.562185 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.562362 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.562605 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.562755 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.562906 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.563050 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.563194 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.563336 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.563496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.563653 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.563798 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.563955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.564111 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.564266 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.564457 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.564633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.565133 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.565297 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.565454 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.565873 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.566030 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.566182 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.566327 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.566478 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.566629 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.566792 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.566946 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.567101 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.567261 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.567414 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.567557 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.567707 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.567850 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.567993 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.568135 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.568292 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.568456 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.568607 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.568751 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.568905 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.569069 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.569069 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.569425 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.569576 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.569720 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.569862 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.570012 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.570164 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.570423 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.570668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.570819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.570980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.571140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.571297 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.571441 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.571585 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.571736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.571889 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.572047 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.572217 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.572459 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.572629 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.572853 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.573005 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.573159 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.573305 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.573466 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.573613 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.573756 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.573906 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.574064 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.574217 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.574367 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.574512 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.574656 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.574813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.574958 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.575107 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.575251 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.575404 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.575548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.575716 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 10:09:21.575871 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.576015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.576170 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.576324 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.576540 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.576702 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.576847 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.576992 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.577142 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.577305 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.577452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.577605 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.577749 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.577901 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.578054 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.578198 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.578351 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.578496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.578647 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.578792 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.578943 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.579094 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.579257 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.579410 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.579555 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.579698 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.579850 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.579994 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.580151 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.580296 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.581110 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.581358 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.581509 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.581653 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.581806 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.581962 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.582114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.582265 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.582429 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.582653 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.582827 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.583045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.583195 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.583346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.583497 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.583657 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.583810 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.583965 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.584109 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.584262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.584428 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.584596 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.584750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.584895 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.585037 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.585189 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.585333 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.585333 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.585621 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.585621 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.585904 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.586047 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.586200 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.586346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.586496 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.586496 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.586778 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.586922 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.587076 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.587221 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.587364 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.587515 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.587659 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.587659 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.588573 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.588735 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.588880 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.589033 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.589178 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.589323 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.589476 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.589629 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.589774 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.589917 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.590068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.590221 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.590384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.590549 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.590764 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.590913 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.591057 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.591215 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.591386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.591618 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.591776 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.591931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.592075 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.592234 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.592380 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.592575 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.592805 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.592956 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.593193 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.593354 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.593501 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.593645 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.593802 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.593954 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.594100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.594258 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.594413 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.594573 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.594717 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.594860 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.595017 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.595162 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.595314 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.595466 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.595619 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.595771 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.595929 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.596074 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.596274 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.596957 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.597115 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.597274 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.597432 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.597585 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.597738 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.597882 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.598025 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.598168 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.598322 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.598473 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.598626 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.598772 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.598916 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.599078 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.599901 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.600068 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.600229 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.600484 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.600653 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.600897 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.601064 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.601224 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.601390 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.601549 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.601824 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.602006 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.602169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.602328 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.602489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.602640 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.602816 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.603045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.603218 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.603456 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.603622 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.603781 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.603943 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.604095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.604255 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.604430 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.604599 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.604765 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.604916 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.604916 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.605211 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.605375 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.605526 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.605675 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.605831 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.605988 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.606152 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.606311 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.606469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.606627 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.606785 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.606935 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.607092 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.607262 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.607421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.607572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.607729 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.607888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.608045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.608222 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.608374 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.608560 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.608713 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.608863 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.609022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.609172 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.609330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.609481 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.609639 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.609790 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.609949 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.610108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.610278 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.610429 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.610579 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.610728 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.610906 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.611133 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.611290 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.611442 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.611593 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.611753 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.612049 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.612216 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.612377 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.612611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.612755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.612904 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.613137 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.613294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.613517 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.613517 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.613754 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.613850 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.613937 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.614016 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.614107 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.614192 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.614282 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.614379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.614465 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.614556 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.614646 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.614738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.614822 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.614913 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.614998 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.615087 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.615179 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.615271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.615364 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.615448 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.615530 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.615620 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.615712 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.615805 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.615897 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.615988 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.616079 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.616163 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.616255 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.616346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.616446 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.616531 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.616614 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.616708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.616807 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.616892 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.616975 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.616975 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.617125 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.617215 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.617301 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.617444 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.617575 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.617713 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.617852 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.617956 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.618042 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.618125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.618209 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.618292 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.618374 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.618466 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.618558 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.618642 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.618733 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.618823 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.618914 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.619009 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.619100 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.619186 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.619268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.619357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.619450 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.619549 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.619636 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.619718 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.619810 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.619904 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.619998 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.620084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.620175 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.620260 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.620350 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.620487 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.620587 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.620672 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.620755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.620837 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.620920 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.621002 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.621129 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.621320 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.621419 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.621504 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.621595 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.621689 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.621775 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.621866 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.621958 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.622069 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.622164 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.622258 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.622350 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.622478 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.622609 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.622791 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.622903 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.623095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.623188 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.623393 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.623499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.623693 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.623794 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.624002 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.624092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.624308 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.624428 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.624641 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.624747 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.624947 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.625039 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.625260 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.625358 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.625560 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.625662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.625862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.625952 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.626172 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.626271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.626476 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.626580 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.626775 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.626914 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.627079 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.627170 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.627381 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.627522 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.627720 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.627857 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.628031 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.628205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.628363 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.628492 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.628662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.628780 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.628974 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.629073 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.629281 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.629379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.629587 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.629686 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.629890 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.629981 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.630193 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.630291 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.630515 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.630633 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.630801 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.630914 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.631103 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.631193 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.631411 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.631560 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.631711 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.631866 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.632006 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.632154 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.632297 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.632503 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.632679 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.632832 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.632985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.633124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.633271 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.633430 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.633568 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.633742 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.633889 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.634037 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.634185 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.634332 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.634549 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.634709 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.634849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.634989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.635134 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.635281 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.635429 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.635575 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.635716 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.635862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.636000 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.636144 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.636296 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.636503 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.636651 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.636792 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.636932 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.637071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.637369 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.637515 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.637677 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.637932 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.638029 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.638123 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.638283 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.638598 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.638908 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.639324 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.639639 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.639950 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.640268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.640611 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.640884 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.641046 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.641218 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.641506 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.641645 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.641824 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.642043 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.642199 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.642422 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.642567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.642814 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.643620 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.643782 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.643928 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.644168 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.644306 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.644484 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.644746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.644853 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.645148 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.645269 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.645547 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.645723 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.645899 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.646177 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.646271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.646637 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.646737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.646964 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.647161 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.647425 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.647548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.647761 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.647899 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.648148 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.648256 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.648510 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.648692 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.648862 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.649015 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.649336 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.649449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.649651 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.649747 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.650098 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.650201 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.650275 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.650363 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.650599 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.650705 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.650917 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.651091 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.651219 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.651397 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.651633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.651731 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.651956 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.652063 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.652271 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.652365 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.652608 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.652701 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.652914 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.653021 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.653227 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.653332 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.653532 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.653629 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.653842 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.653938 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.654149 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.654245 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.654455 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.654559 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.654759 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.654870 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.655070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.655201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.655376 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.655517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.655679 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.655831 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.655987 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.656139 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.656290 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.656475 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.656632 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.656930 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.657036 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.657234 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.657336 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.657539 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.657638 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.657850 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.657950 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.658160 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.658257 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.658465 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.658563 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.659327 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.659490 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.659637 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.659802 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.659944 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.660111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.660248 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.660439 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.660581 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.660753 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.660884 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.661077 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.661178 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.661385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.661484 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.661787 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.661879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.662097 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.662195 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.662397 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.662488 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.662706 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.662812 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.663020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.663119 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.663308 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.663489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.663609 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.663812 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.663915 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.664124 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.664221 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.664512 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.664623 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.664813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.664919 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.665115 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.665212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.665416 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.665513 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.665734 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.665886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.666040 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.666210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.666345 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.666519 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.666651 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.666823 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.666960 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.667135 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.667298 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.667463 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.667601 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.667788 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.667956 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.668099 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.668300 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.668440 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.668593 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.668726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.668914 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.669052 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.669224 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.669372 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.669531 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.669686 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.669847 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.670011 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.670148 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.670421 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.670515 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.670742 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.670842 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.671049 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.671155 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.671353 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.671452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.671654 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.671804 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.671971 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.672125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.672457 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.672552 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.672772 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.673076 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.673384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.673475 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.673707 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.673820 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.674015 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.674152 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.674354 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.674510 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.674661 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.674821 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.674968 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.675136 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.675271 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.675448 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.675573 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.675759 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.675894 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.676072 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.676197 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.677327 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.677439 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.677439 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.677593 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.677593 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.677767 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.677853 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.677944 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.677944 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.678099 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.678178 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.678271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.678381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.678470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.678625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.678744 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.678928 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.679032 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.679230 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.679340 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.679571 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.679679 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.679874 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.679972 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.680193 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.680304 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.680523 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.680831 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.681136 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.681533 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.681646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.681794 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.681899 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.682343 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.682455 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.682659 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.682813 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.682967 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.683083 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.683275 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.683405 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.683582 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.683734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.683896 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.684052 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.684210 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.684355 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.684541 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.684693 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.684958 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.685065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.685265 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.685366 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.685573 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.685680 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.685879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.685970 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.686179 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.686270 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.686480 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.686581 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.686785 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.686885 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.687105 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.687218 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.687411 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.687563 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.687712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.687887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.688016 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.688198 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.688336 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.688530 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.688681 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.688838 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.688983 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.689145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.689398 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.689498 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.689707 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.689804 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.690541 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.690959 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.691209 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.691306 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.691530 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.691704 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.691832 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.692039 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.692154 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.692350 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.692510 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.692700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.692810 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.693018 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.693112 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.693335 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.693434 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.693651 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.693751 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.693965 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.694058 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.694295 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.694463 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.694603 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.694787 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.694916 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.695104 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.695221 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.695416 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.695542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.695731 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.695863 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.696046 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.696176 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.696362 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.696520 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.696937 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.697037 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.697259 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.697360 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.697568 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.697669 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.697883 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.697974 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.698199 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.698291 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.698502 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.698601 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.698806 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.698918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.699115 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.699228 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.699421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.699531 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.699723 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.699824 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.700033 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.700132 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.700332 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.700477 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.700671 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.700785 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.700981 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.701095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.701284 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.701386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.701585 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.701685 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.701886 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.702079 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.702206 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.702397 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.702491 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.702713 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.702813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.703071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.703180 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.703389 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.703483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.703700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.703801 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.703999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.704105 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.704319 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.704443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.704650 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.704739 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.704953 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.705058 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.705252 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.705345 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.705585 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.706164 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.706424 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.706525 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.706734 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.706835 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.707054 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.707146 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.707383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.707483 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.707713 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.707807 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.708101 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.708206 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.708473 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.708573 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.708848 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.708964 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.709113 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.709222 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.709417 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.709519 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.709735 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.709890 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.710251 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.710363 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.710568 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.710674 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.710878 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.710982 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.711183 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.711296 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.711490 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.711589 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.711793 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.711897 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.712102 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.712212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.712429 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.712524 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.712730 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.712822 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.713030 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.713135 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.713351 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.713448 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.713654 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.713759 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.713974 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.714071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.714282 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.714380 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.714590 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.714683 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.714892 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.714994 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.715239 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.715356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.715540 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.715646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.715957 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.716119 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.716291 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.716524 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.716737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.716848 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.717072 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.717160 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.717459 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.717571 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.717774 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.717882 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.718084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.718226 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.718405 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.718588 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.718704 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.718909 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.719010 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.719224 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.719319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.719538 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.719638 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.719853 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.719953 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.720168 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.720269 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.720500 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.720683 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.720810 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.721106 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.721230 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.721440 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.721533 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.721742 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.721862 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.722065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.722329 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.722421 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.722660 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.722927 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.723259 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.723359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.723577 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.723669 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.723880 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.723980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.724197 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.724301 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.724536 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.724845 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.725152 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.725267 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.725601 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.725718 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.725926 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.726029 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.726249 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.726343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.726568 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.726662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.726883 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.726974 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.727214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.727302 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.727606 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.727717 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.727937 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.728119 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.728279 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.728532 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.728642 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.728862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.729013 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.729171 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.729490 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.729803 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.730096 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.730203 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.730476 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.730591 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.730822 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.730956 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.731359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.731682 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.731792 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.731881 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.731995 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.732088 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.732323 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.732589 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.732698 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.732920 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.733021 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.733245 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.733338 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.733548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.733652 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.733866 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.733966 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.734175 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.734289 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.734487 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.734599 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.734793 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.734889 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.735157 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.735274 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.735409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.735515 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.735718 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.735825 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.736044 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.736141 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.736379 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.736494 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.736731 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.736850 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.737099 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.737206 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.737476 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.737586 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.737732 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.737849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.738080 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.738254 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.738394 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.738571 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.738729 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.738906 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.739045 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.739248 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.739442 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.739554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.739759 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.739860 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.740074 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.740179 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.740383 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.740523 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.740710 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.740802 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.741018 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.741119 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.741325 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.741417 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.741637 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.741729 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.741963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.742065 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.742273 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.742365 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.742574 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.742665 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.742874 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.742974 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.743194 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.743295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.743504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.743611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.743884 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.744033 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.744196 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.744377 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.744528 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.744714 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.744844 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.745037 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.745153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.745363 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.745475 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.745680 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.745781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.745989 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.746088 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.746316 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.746438 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.746618 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.746709 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.746933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.747046 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.747244 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.747349 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.747554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.747642 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.747858 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.747952 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.748164 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.748272 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.748498 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.748609 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.748804 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.748903 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.749104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.749203 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.749404 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.749504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.749701 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.749793 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.749997 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.750096 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.750299 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.750391 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.750645 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.750753 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.750903 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.751002 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.751207 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.751297 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.751506 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.751605 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.751803 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.751897 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.752104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.752196 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.752918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.753031 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.753246 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.753341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.753550 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.753648 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.753855 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.753947 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.754154 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.754246 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.754451 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.754543 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.754752 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.754855 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.755050 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.755141 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.755353 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.755453 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.755679 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.755872 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.756010 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.756213 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.756304 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.756616 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.756715 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.756934 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.757018 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.757276 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.757365 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.757583 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.757665 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.757893 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.757975 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.758202 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.758291 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.758509 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.758599 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.758815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.758896 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.759141 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.759365 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.759456 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.759710 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.759831 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.760031 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.760147 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.760487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.760583 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.760719 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.760812 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.761030 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.761120 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.761340 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.761431 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.761650 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.761731 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.761970 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.762070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.762279 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.762381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.762590 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.762682 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.762901 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.763001 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.763209 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.763308 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.763514 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.763612 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.763821 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.763912 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.764129 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.764222 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.764455 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.764587 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.764779 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.764884 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.765095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.765194 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.765401 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.765514 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.765708 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.766603 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.766746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.766825 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.766898 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.766968 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.767047 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.767120 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.767204 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.767289 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.767369 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.767467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.767640 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.767850 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.767987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.768887 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.768992 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.769209 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.769315 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.769525 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.769633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.769841 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.769955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.770150 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.770242 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.770467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.770568 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.770786 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117813) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 05 10:09:21.770880 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.771100 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.771407 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.771829 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.772143 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.772484 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.772808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.773125 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.773450 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.773774 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.774088 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.774410 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.774722 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.775052 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.775392 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.775723 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.776049 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.776356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.776688 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.776996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.777313 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.777633 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.777950 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.778257 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.778561 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.778870 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.779175 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.779478 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.779783 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.780089 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.780429 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.780737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.781030 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.781332 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.781636 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.781938 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.782243 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.782551 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.782857 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.783162 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.783464 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.783809 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.784115 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.784487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.784824 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.785140 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.785462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.785774 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.786110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.786398 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.786720 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.787043 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.787368 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.787679 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.788011 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.788324 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.788858 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.789160 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.789562 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.789878 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.790834 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.790834 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.790834 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.791135 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.791507 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.792013 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.792347 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.792693 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.793008 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.793340 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.793653 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.793964 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.794352 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.794588 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.794901 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.795215 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.795532 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.795863 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.796186 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.796681 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.797014 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.797348 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.797701 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.798019 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.798369 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.798699 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.799016 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.799372 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.799685 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 10:09:21.799930 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117812) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 05 10:09:21.926865 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO dbcounter [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] Registered counter for database nova_api Jul 05 10:09:21.949621 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_db.sqlalchemy.engines [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117813) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:09:21.954903 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO dbcounter [None req-3c705435-16b3-45df-a624-20110d63f39a None None] Registered counter for database nova_api Jul 05 10:09:21.957309 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writer thread running {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:09:21.975068 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_db.sqlalchemy.engines [None req-3c705435-16b3-45df-a624-20110d63f39a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117812) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:09:21.982127 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writer thread running {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:09:22.016734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.context [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),21c70492-35f8-4468-a997-987c705bd844(cell1) {{(pid=117813) load_cells /opt/stack/nova/nova/context.py:464}} Jul 05 10:09:22.021139 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:09:22.022701 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:09:22.023510 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:09:22.025513 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:09:22.027477 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO dbcounter [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] Registered counter for database nova_cell0 Jul 05 10:09:22.031554 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO dbcounter [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] Registered counter for database nova_cell1 Jul 05 10:09:22.040589 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_db.sqlalchemy.engines [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117813) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:09:22.042841 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_db.sqlalchemy.engines [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117813) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:09:22.043805 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.context [None req-3c705435-16b3-45df-a624-20110d63f39a None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),21c70492-35f8-4468-a997-987c705bd844(cell1) {{(pid=117812) load_cells /opt/stack/nova/nova/context.py:464}} Jul 05 10:09:22.047785 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3c705435-16b3-45df-a624-20110d63f39a None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:09:22.048546 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writer thread running {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:09:22.049413 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3c705435-16b3-45df-a624-20110d63f39a None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:09:22.050223 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3c705435-16b3-45df-a624-20110d63f39a None None] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:09:22.050393 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writer thread running {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:09:22.052161 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3c705435-16b3-45df-a624-20110d63f39a None None] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:09:22.054232 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO dbcounter [None req-3c705435-16b3-45df-a624-20110d63f39a None None] Registered counter for database nova_cell0 Jul 05 10:09:22.058374 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO dbcounter [None req-3c705435-16b3-45df-a624-20110d63f39a None None] Registered counter for database nova_cell1 Jul 05 10:09:22.065416 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_db.sqlalchemy.engines [None req-3c705435-16b3-45df-a624-20110d63f39a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117812) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:09:22.068061 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_db.sqlalchemy.engines [None req-3c705435-16b3-45df-a624-20110d63f39a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117812) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:09:22.074989 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writer thread running {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:09:22.076474 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writer thread running {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:09:22.196501 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO dbcounter [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] Registered counter for database nova_cell0 Jul 05 10:09:22.203212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_db.sqlalchemy.engines [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117813) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:09:22.208285 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writer thread running {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:09:22.219912 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO dbcounter [None req-3c705435-16b3-45df-a624-20110d63f39a None None] Registered counter for database nova_cell0 Jul 05 10:09:22.227119 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_db.sqlalchemy.engines [None req-3c705435-16b3-45df-a624-20110d63f39a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117812) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 10:09:22.231480 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writer thread running {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 10:09:22.526356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: WARNING keystonemiddleware.auth_token [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 05 10:09:22.533793 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: WARNING keystonemiddleware.auth_token [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 05 10:09:22.553386 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: WARNING keystonemiddleware.auth_token [None req-3c705435-16b3-45df-a624-20110d63f39a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 05 10:09:22.557690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: WARNING keystonemiddleware.auth_token [None req-3c705435-16b3-45df-a624-20110d63f39a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 05 10:09:22.874221 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: WARNING keystonemiddleware.auth_token [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 05 10:09:22.880825 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: WARNING keystonemiddleware.auth_token [None req-c2f50153-8cd7-4511-83c6-de0fd7c864d6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 05 10:09:22.886090 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5581072f69c0 pid: 117813 (default app) Jul 05 10:09:22.898036 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: WARNING keystonemiddleware.auth_token [None req-3c705435-16b3-45df-a624-20110d63f39a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 05 10:09:22.904145 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: WARNING keystonemiddleware.auth_token [None req-3c705435-16b3-45df-a624-20110d63f39a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 05 10:09:22.909472 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5581072f69c0 pid: 117812 (default app) Jul 05 10:10:02.713411 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=3 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:02.714237 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=3 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:02.715145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=3 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:02.716094 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=3 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:03.527634 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-014611f2-221e-459b-9716-dfc07b2413c1 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Action: 'create', calling method: .version_select at 0x7feb8b680550>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1245070540"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:03.917417 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=3 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:03.918534 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=3 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:03.919317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=3 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:03.922318 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=3 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:05.402615 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ee263349-32ec-4525-a47b-fec3c542338c tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] Action: 'create', calling method: .version_select at 0x7feb8b681550>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-9821728", "metadata": {"Type": "work"}}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:06.832422 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ee263349-32ec-4525-a47b-fec3c542338c tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] 10.222.0.20 "POST /compute/v2.1/os-volumes" status: 200 len: 350 microversion: 2.1 time: 2.919260 Jul 05 10:10:06.833726 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 1/1] 10.222.0.20 () {60 vars in 1155 bytes} [Tue Jul 5 10:10:03 2022] POST /compute/v2.1/os-volumes => generated 350 bytes in 2923 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Jul 05 10:10:06.848503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-17d86dd1-36c7-435f-a132-6fa614ea32c7 tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] Calling method '.version_select at 0x7feb8b545430>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:06.899919 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-014611f2-221e-459b-9716-dfc07b2413c1 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 4.204338 Jul 05 10:10:06.900742 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 1/2] 10.222.0.20 () {60 vars in 1154 bytes} [Tue Jul 5 10:10:02 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 4207 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Jul 05 10:10:06.914834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d6fb9150-4767-4c38-b5a6-11662496fd71 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8b548430>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:07.020868 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-17d86dd1-36c7-435f-a132-6fa614ea32c7 tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/a8257fa4-da28-4a74-bf06-afbca1582440" status: 200 len: 350 microversion: 2.1 time: 0.178540 Jul 05 10:10:07.022450 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:06 2022] GET /compute/v2.1/os-volumes/a8257fa4-da28-4a74-bf06-afbca1582440 => generated 350 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:10:07.116965 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d6fb9150-4767-4c38-b5a6-11662496fd71 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124" status: 200 len: 345 microversion: 2.1 time: 0.206924 Jul 05 10:10:07.117852 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:06 2022] GET /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124 => generated 345 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:07.717871 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e8a8c3a7-0e34-470b-a32d-62fe00a1dc13 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] Calling method '.version_select at 0x7feb8b545790>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:07.860778 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e8a8c3a7-0e34-470b-a32d-62fe00a1dc13 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/7c3a02ad-33d7-4240-b1a9-b79e09497df7" status: 200 len: 351 microversion: 2.1 time: 0.150178 Jul 05 10:10:07.861409 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 3/5] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:07 2022] GET /compute/v2.1/os-volumes/7c3a02ad-33d7-4240-b1a9-b79e09497df7 => generated 351 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:08.034743 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c483426e-5432-47c1-b5d4-035b7eff4f61 tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] Calling method '.version_select at 0x7feb8b548550>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:08.130949 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3fdb98cd-0498-4f7e-91fc-8d1e53fe9650 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8b545c10>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:08.210148 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c483426e-5432-47c1-b5d4-035b7eff4f61 tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/a8257fa4-da28-4a74-bf06-afbca1582440" status: 200 len: 351 microversion: 2.1 time: 0.179986 Jul 05 10:10:08.211108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 3/6] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:08 2022] GET /compute/v2.1/os-volumes/a8257fa4-da28-4a74-bf06-afbca1582440 => generated 351 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:08.223786 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6b9d9ae7-4c5e-4dff-8160-542aaa6ccd92 tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] Calling method '.version_select at 0x7feb8b548c10>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:08.567041 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-6b9d9ae7-4c5e-4dff-8160-542aaa6ccd92 tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/a8257fa4-da28-4a74-bf06-afbca1582440" status: 200 len: 351 microversion: 2.1 time: 0.347461 Jul 05 10:10:08.569772 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:08 2022] GET /compute/v2.1/os-volumes/a8257fa4-da28-4a74-bf06-afbca1582440 => generated 351 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:08.587820 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a0c0648f-b9d0-4d84-b6fe-c369cb4fa06f tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] Calling method '.version_select at 0x7feb8b563310>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:08.674267 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3fdb98cd-0498-4f7e-91fc-8d1e53fe9650 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124" status: 200 len: 346 microversion: 2.1 time: 0.548244 Jul 05 10:10:08.676273 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:08 2022] GET /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124 => generated 346 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:08.694919 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d199859c-7157-4b04-8bff-0d6003d66754 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Action: 'create', calling method: .version_select at 0x7feb8b55f310>, body: {"snapshot": {"volume_id": "516dd2b4-ea0e-4117-91b0-ac7e38b19124", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1580806028"}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:08.901350 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a0c0648f-b9d0-4d84-b6fe-c369cb4fa06f tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] 10.222.0.20 "DELETE /compute/v2.1/os-volumes/a8257fa4-da28-4a74-bf06-afbca1582440" status: 202 len: 0 microversion: 2.1 time: 0.315797 Jul 05 10:10:08.905101 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 5/9] 10.222.0.20 () {58 vars in 1247 bytes} [Tue Jul 5 10:10:08 2022] DELETE /compute/v2.1/os-volumes/a8257fa4-da28-4a74-bf06-afbca1582440 => generated 0 bytes in 319 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:10:08.912492 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1a16829c-2b24-48d2-9347-ddaf97082b6a tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] Calling method '.version_select at 0x7feb8b548dc0>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:09.197121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1a16829c-2b24-48d2-9347-ddaf97082b6a tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/a8257fa4-da28-4a74-bf06-afbca1582440" status: 200 len: 350 microversion: 2.1 time: 0.290729 Jul 05 10:10:09.198792 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 6/10] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:08 2022] GET /compute/v2.1/os-volumes/a8257fa4-da28-4a74-bf06-afbca1582440 => generated 350 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:09.546980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d199859c-7157-4b04-8bff-0d6003d66754 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.862588 Jul 05 10:10:09.547831 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 5/11] 10.222.0.20 () {60 vars in 1161 bytes} [Tue Jul 5 10:10:08 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 864 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:09.558678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e20d8118-21b9-4bb9-aa9e-f2ef933a0a4d tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8b5484c0>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:09.662266 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e20d8118-21b9-4bb9-aa9e-f2ef933a0a4d tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04" status: 200 len: 287 microversion: 2.1 time: 0.107574 Jul 05 10:10:09.663123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 7/12] 10.222.0.20 () {58 vars in 1250 bytes} [Tue Jul 5 10:10:09 2022] GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 => generated 287 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:09.677741 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e1e30412-d9e9-42dc-b120-d6d3aea4d996 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8b545e50>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:09.816657 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e1e30412-d9e9-42dc-b120-d6d3aea4d996 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.147114 Jul 05 10:10:09.817310 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 6/13] 10.222.0.20 () {58 vars in 1139 bytes} [Tue Jul 5 10:10:09 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:09.830343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-dc1d5c12-716e-4bf9-86c1-e91b44a6e839 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8b563430>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:09.979900 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-992c1d39-686f-4438-8391-241f00901dd5 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] Calling method '.version_select at 0x7feb8b55f3a0>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:10.071306 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-dc1d5c12-716e-4bf9-86c1-e91b44a6e839 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04" status: 200 len: 287 microversion: 2.1 time: 0.246522 Jul 05 10:10:10.071882 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 8/14] 10.222.0.20 () {58 vars in 1250 bytes} [Tue Jul 5 10:10:09 2022] GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 => generated 287 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:10.185580 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-992c1d39-686f-4438-8391-241f00901dd5 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/b846d0e6-0b44-4cdc-812f-344273fe87c5" status: 200 len: 350 microversion: 2.1 time: 0.210326 Jul 05 10:10:10.186488 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 7/15] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:09 2022] GET /compute/v2.1/os-volumes/b846d0e6-0b44-4cdc-812f-344273fe87c5 => generated 350 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:10:10.215801 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2d963ef7-2cf7-4ea9-83f7-405671f6e501 tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] Calling method '.version_select at 0x7feb8b548700>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:10.291363 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-2d963ef7-2cf7-4ea9-83f7-405671f6e501 tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] HTTP exception thrown: Volume a8257fa4-da28-4a74-bf06-afbca1582440 could not be found. Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2d963ef7-2cf7-4ea9-83f7-405671f6e501 tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] Request method failure captured: Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/os-volumes/a8257fa4-da28-4a74-bf06-afbca1582440 HTTP/1.1 Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA37s187bYmbFLRg6lU1y-xAqn7pvcasonPMcFoim_hAxg13FQUEdfCcEa4kfMmZemDHKtiR9tjgI9w9hwarqOv7U2_yvmvT06q8ZIoUpeADv0BXJrC1sbAye8j0q2lfpQAz3_k1PVDYmfi1S3ojAjPXQWx-mtfECeGr-2_XeWviD-c Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 38a04b3007354681be79ffb9cdc8d355 Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-VolumesGetTestJSON-1272635986 Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:10:10.301359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:10:10.302517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/38a04b3007354681be79ffb9cdc8d355"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_38a04b3007354681be79ffb9cdc8d355"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/38a04b3007354681be79ffb9cdc8d355"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/38a04b3007354681be79ffb9cdc8d355"}]}] Jul 05 10:10:10.302517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-VolumesGetTestJSON-1272635986 Jul 05 10:10:10.302517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 38a04b3007354681be79ffb9cdc8d355 Jul 05 10:10:10.302517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-VolumesGetTestJSON-1272635986 Jul 05 10:10:10.302517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-VolumesGetTestJSON-1272635986-project Jul 05 10:10:10.302517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:10:10.302517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:10:10.302517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: dd8be32629574f6fbda0f0f141ee8b16 Jul 05 10:10:10.302517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-VolumesGetTestJSON-1272635986-project Jul 05 10:10:10.302517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: Jul 05 10:10:10.302517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Volume a8257fa4-da28-4a74-bf06-afbca1582440 could not be found. Jul 05 10:10:10.302517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'a8257fa4-da28-4a74-bf06-afbca1582440'} Jul 05 10:10:10.302517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume a8257fa4-da28-4a74-bf06-afbca1582440 could not be found. (HTTP 404) (Request-ID: req-5843a22b-c365-4ac5-ab10-0f8b66be1aa5) Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:10.303183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 05 10:10:10.304095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume a8257fa4-da28-4a74-bf06-afbca1582440 could not be found. Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 05 10:10:10.305003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:10:10.305862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume a8257fa4-da28-4a74-bf06-afbca1582440 could not be found. Jul 05 10:10:10.305862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:10.305862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:10.305862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:10.305862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:10.305862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:10.305862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:10.305862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:10:10.305862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:10:10.305862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume a8257fa4-da28-4a74-bf06-afbca1582440 could not be found. Jul 05 10:10:10.305862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:10.305862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2d963ef7-2cf7-4ea9-83f7-405671f6e501 tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] Returning 404 to user: Volume a8257fa4-da28-4a74-bf06-afbca1582440 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:10:10.305862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2d963ef7-2cf7-4ea9-83f7-405671f6e501 tempest-VolumesGetTestJSON-1272635986 tempest-VolumesGetTestJSON-1272635986-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/a8257fa4-da28-4a74-bf06-afbca1582440" status: 404 len: 109 microversion: 2.1 time: 0.096900 Jul 05 10:10:10.306465 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 9/16] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:10 2022] GET /compute/v2.1/os-volumes/a8257fa4-da28-4a74-bf06-afbca1582440 => generated 109 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:10:11.084659 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2ac61cf0-45a6-4b25-8ef9-9854396abe44 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8b55f5e0>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:11.170891 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2ac61cf0-45a6-4b25-8ef9-9854396abe44 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04" status: 200 len: 287 microversion: 2.1 time: 0.092423 Jul 05 10:10:11.171864 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 8/17] 10.222.0.20 () {58 vars in 1250 bytes} [Tue Jul 5 10:10:11 2022] GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 => generated 287 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:12.189327 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c9f2e988-453c-4eb6-a655-a23ca1984331 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8bf289d0>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:12.277306 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c9f2e988-453c-4eb6-a655-a23ca1984331 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04" status: 200 len: 287 microversion: 2.1 time: 0.096341 Jul 05 10:10:12.278436 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 10/18] 10.222.0.20 () {58 vars in 1250 bytes} [Tue Jul 5 10:10:12 2022] GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 => generated 287 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:13.289885 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-43647ae3-17f0-4dc5-8a3d-dad48a372517 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8b545790>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:13.342665 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-43647ae3-17f0-4dc5-8a3d-dad48a372517 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04" status: 200 len: 287 microversion: 2.1 time: 0.057083 Jul 05 10:10:13.343238 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 9/19] 10.222.0.20 () {58 vars in 1250 bytes} [Tue Jul 5 10:10:13 2022] GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 => generated 287 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:13.845500 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1862934180", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "networks": [{"uuid": "91d9a322-3f41-4d40-8000-86bec9aba65e"}]}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:14.081043 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.network.neutron [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] validate_networks() for [('91d9a322-3f41-4d40-8000-86bec9aba65e', None, None, None, None, None)] {{(pid=117813) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:10:14.357509 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ec2d6c27-edbd-466a-8a8a-5b3a9424d68c tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8b545c10>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:14.406876 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ec2d6c27-edbd-466a-8a8a-5b3a9424d68c tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04" status: 200 len: 287 microversion: 2.1 time: 0.054897 Jul 05 10:10:14.411679 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 10/20] 10.222.0.20 () {58 vars in 1250 bytes} [Tue Jul 5 10:10:14 2022] GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 => generated 287 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:15.419134 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-623333b7-badd-427e-8db5-c098c985c709 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8b55f4c0>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:15.494072 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-623333b7-badd-427e-8db5-c098c985c709 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04" status: 200 len: 287 microversion: 2.1 time: 0.079927 Jul 05 10:10:15.498913 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 11/21] 10.222.0.20 () {58 vars in 1250 bytes} [Tue Jul 5 10:10:15 2022] GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 => generated 287 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:10:15.980936 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Flavor limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:10:15.982042 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Image limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:10:15.983058 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Flavor pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:10:15.983669 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Image pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:10:15.984528 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:10:16.034976 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:16.042320 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:16.044380 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:16.045517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:16.138367 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Getting quotas for project f1f4fee9c6b844a2b7a14e2afb831f88. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:10:16.164354 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Getting quotas for user a9d95652b0714a2a8427a7e99de27bee and project f1f4fee9c6b844a2b7a14e2afb831f88. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:10:16.204884 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Going to run 1 instances... {{(pid=117813) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:10:16.212644 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117813) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:10:16.372033 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-bbf66b7c-0779-45d1-ad7a-63202f46bb58 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.957921 Jul 05 10:10:16.373810 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 11/22] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:10:13 2022] POST /compute/v2.1/servers => generated 380 bytes in 2960 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:10:16.385970 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-738f7065-bd12-4b8a-a8b0-d3bab01c0a08 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:16.386920 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-738f7065-bd12-4b8a-a8b0-d3bab01c0a08 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:16.508510 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-80b945fd-f865-4b42-a1f0-59f7bb31a05c tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8abd1160>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:16.582308 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-80b945fd-f865-4b42-a1f0-59f7bb31a05c tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04" status: 200 len: 287 microversion: 2.1 time: 0.079050 Jul 05 10:10:16.584629 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 12/23] 10.222.0.20 () {58 vars in 1250 bytes} [Tue Jul 5 10:10:16 2022] GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 => generated 287 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:16.650854 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-738f7065-bd12-4b8a-a8b0-d3bab01c0a08 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:16.683401 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-738f7065-bd12-4b8a-a8b0-d3bab01c0a08 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1215 microversion: 2.1 time: 0.303078 Jul 05 10:10:16.686514 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 12/24] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:16 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1215 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:17.498165 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-97b2b726-d8e3-4893-a3e0-51ee863ab618 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] Calling method '.version_select at 0x7feb8abb1ee0>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:17.600532 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-62eabbe9-1ad0-4bbb-9c6f-b5116489328d tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8b56fe50>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:17.607814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-97b2b726-d8e3-4893-a3e0-51ee863ab618 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/12cd64d5-1498-4106-9683-c9dd0e18a1cc" status: 200 len: 351 microversion: 2.1 time: 0.113457 Jul 05 10:10:17.608859 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 13/25] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:17 2022] GET /compute/v2.1/os-volumes/12cd64d5-1498-4106-9683-c9dd0e18a1cc => generated 351 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:17.627765 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8cbdf488-94be-4eb9-bc2b-c493aa997f0e tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] Calling method '.version_select at 0x7feb8abd15e0>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:17.696588 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-62eabbe9-1ad0-4bbb-9c6f-b5116489328d tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04" status: 200 len: 288 microversion: 2.1 time: 0.104615 Jul 05 10:10:17.697395 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 13/26] 10.222.0.20 () {58 vars in 1250 bytes} [Tue Jul 5 10:10:17 2022] GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 => generated 288 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:17.710472 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-434026fd-1f7a-4a24-994a-003d0e523586 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8b5281f0>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:17.796455 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8cbdf488-94be-4eb9-bc2b-c493aa997f0e tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] 10.222.0.20 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.179712 Jul 05 10:10:17.797112 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 14/27] 10.222.0.20 () {58 vars in 1133 bytes} [Tue Jul 5 10:10:17 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:17.801905 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e3cc325e-f27a-4ed7-a381-4419eecb96d5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:17.802579 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e3cc325e-f27a-4ed7-a381-4419eecb96d5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:17.838130 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-434026fd-1f7a-4a24-994a-003d0e523586 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "DELETE /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04" status: 202 len: 0 microversion: 2.1 time: 0.133795 Jul 05 10:10:17.838706 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 14/28] 10.222.0.20 () {58 vars in 1253 bytes} [Tue Jul 5 10:10:17 2022] DELETE /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 => generated 0 bytes in 135 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 05 10:10:17.853726 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c3b98ccb-0c4b-40ba-accb-5e45af153cba tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] Calling method '.version_select at 0x7feb8b5455e0>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:17.936229 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e3cc325e-f27a-4ed7-a381-4419eecb96d5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:17.966433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e3cc325e-f27a-4ed7-a381-4419eecb96d5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1215 microversion: 2.1 time: 0.167533 Jul 05 10:10:17.978040 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 15/29] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:17 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1215 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:17.990268 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-43cbd26b-cb5e-401c-98d3-b8807c97a218 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8abd1f70>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:18.015172 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-c3b98ccb-0c4b-40ba-accb-5e45af153cba tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] 10.222.0.20 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.175121 Jul 05 10:10:18.016228 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 15/30] 10.222.0.20 () {58 vars in 1148 bytes} [Tue Jul 5 10:10:17 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:18.030520 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b0197148-cb8d-4883-a0f0-60eacbc2ef13 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] Calling method '.version_select at 0x7feb8b528700>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:18.081189 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-43cbd26b-cb5e-401c-98d3-b8807c97a218 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04" status: 200 len: 287 microversion: 2.1 time: 0.103253 Jul 05 10:10:18.082289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 16/31] 10.222.0.20 () {58 vars in 1250 bytes} [Tue Jul 5 10:10:17 2022] GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 => generated 287 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:18.187026 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b0197148-cb8d-4883-a0f0-60eacbc2ef13 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] 10.222.0.20 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.161827 Jul 05 10:10:18.188084 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 16/32] 10.222.0.20 () {58 vars in 1133 bytes} [Tue Jul 5 10:10:18 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:18.202032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1969a256-8742-4b23-9edc-d0e374f83497 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] Calling method '.version_select at 0x7feb8a373670>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:18.370610 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1969a256-8742-4b23-9edc-d0e374f83497 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] 10.222.0.20 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.174372 Jul 05 10:10:18.371291 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 17/33] 10.222.0.20 () {58 vars in 1166 bytes} [Tue Jul 5 10:10:18 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:18.384968 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0dbf1345-368d-4e63-9138-c4056c011146 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] Calling method '.version_select at 0x7feb8b52e3a0>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:18.506108 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0dbf1345-368d-4e63-9138-c4056c011146 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.126851 Jul 05 10:10:18.508097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 17/34] 10.222.0.20 () {58 vars in 1169 bytes} [Tue Jul 5 10:10:18 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:18.523435 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ad30e385-3187-4748-83c8-a1caf02035e7 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] Calling method '.version_select at 0x7feb8a37d310>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:18.895181 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ad30e385-3187-4748-83c8-a1caf02035e7 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.376049 Jul 05 10:10:18.907354 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 18/35] 10.222.0.20 () {58 vars in 1154 bytes} [Tue Jul 5 10:10:18 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:18.908530 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f32fa8db-b8bd-4e1b-8549-83ee026f3be0 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] Calling method '.version_select at 0x7feb8b5289d0>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:18.996155 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-98d6f95b-97f4-43a8-99c7-51894ec47b68 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:18.998228 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-98d6f95b-97f4-43a8-99c7-51894ec47b68 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:19.031924 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-98d6f95b-97f4-43a8-99c7-51894ec47b68 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:19.033567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-98d6f95b-97f4-43a8-99c7-51894ec47b68 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:19.035797 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-98d6f95b-97f4-43a8-99c7-51894ec47b68 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:19.036945 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-98d6f95b-97f4-43a8-99c7-51894ec47b68 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:19.200201 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f32fa8db-b8bd-4e1b-8549-83ee026f3be0 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.296055 Jul 05 10:10:19.201155 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 18/36] 10.222.0.20 () {58 vars in 1187 bytes} [Tue Jul 5 10:10:18 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:19.206548 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7020e419-f2b8-41ce-a134-8e4523838e71 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8bf27790>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:19.274565 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-7020e419-f2b8-41ce-a134-8e4523838e71 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] HTTP exception thrown: Snapshot b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 could not be found. Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7020e419-f2b8-41ce-a134-8e4523838e71 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Request method failure captured: Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 HTTP/1.1 Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA361yXo1lbM2IMddZmvV505MNtBrbX23Zw-4dNEeqvOsreYY1jfHS0RSRhOEMdg7oIW-szddpKuBvkIvsX_df2UbGsOfvlPrAsZ8UZ8GY9SBEsoCt5ZjX8yCR4S5AeHedMPWPFSgGwjRj6gPvInXrHhSInOWqoposxqPKKNDo6Epbs Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: a75ede11bffb460aa6cb57e232cfa4d4 Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1214286970 Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:10:19.278999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:10:19.280028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/a75ede11bffb460aa6cb57e232cfa4d4"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_a75ede11bffb460aa6cb57e232cfa4d4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/a75ede11bffb460aa6cb57e232cfa4d4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/a75ede11bffb460aa6cb57e232cfa4d4"}]}] Jul 05 10:10:19.280028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1214286970 Jul 05 10:10:19.280028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: a75ede11bffb460aa6cb57e232cfa4d4 Jul 05 10:10:19.280028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1214286970 Jul 05 10:10:19.280028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-VolumesSnapshotsTestJSON-1214286970-project Jul 05 10:10:19.280028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:10:19.280028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:10:19.280028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: eca9bd479a194bfaabb8135605b8bf26 Jul 05 10:10:19.280028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1214286970-project Jul 05 10:10:19.280028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: Jul 05 10:10:19.280028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Snapshot b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 could not be found. Jul 05 10:10:19.280028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': 'b898f1f4-9b0d-46c2-8f26-51c4afe1ec04'} Jul 05 10:10:19.280028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 could not be found. (HTTP 404) (Request-ID: req-0ebb1635-f7db-4afe-8f98-0c36426ae2f9) Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:19.280690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 05 10:10:19.281504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 could not be found. Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Jul 05 10:10:19.282402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:10:19.283317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 could not be found. Jul 05 10:10:19.283317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:19.283317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:19.283317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:19.283317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:19.283317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:19.283317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:19.283317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:10:19.283317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:10:19.283317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 could not be found. Jul 05 10:10:19.283317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:19.283317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7020e419-f2b8-41ce-a134-8e4523838e71 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Returning 404 to user: Snapshot b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:10:19.283317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7020e419-f2b8-41ce-a134-8e4523838e71 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04" status: 404 len: 111 microversion: 2.1 time: 0.078170 Jul 05 10:10:19.283918 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 19/37] 10.222.0.20 () {58 vars in 1250 bytes} [Tue Jul 5 10:10:19 2022] GET /compute/v2.1/os-snapshots/b898f1f4-9b0d-46c2-8f26-51c4afe1ec04 => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:10:19.290188 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2701fda4-b2ad-44ac-aa01-fdf9f077c149 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] Calling method '.version_select at 0x7feb8b52e820>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:19.473310 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-98d6f95b-97f4-43a8-99c7-51894ec47b68 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:19.498525 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-98d6f95b-97f4-43a8-99c7-51894ec47b68 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1267 microversion: 2.1 time: 0.518167 Jul 05 10:10:19.505323 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 19/38] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:18 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1267 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:19.515575 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e2c8bbf9-348f-4c12-b48e-c7092b816d46 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8a2f0b80>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:19.518107 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2701fda4-b2ad-44ac-aa01-fdf9f077c149 tempest-VolumesTestJSON-1774113168 tempest-VolumesTestJSON-1774113168-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.235601 Jul 05 10:10:19.519437 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 20/39] 10.222.0.20 () {58 vars in 1154 bytes} [Tue Jul 5 10:10:19 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:19.682173 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e2c8bbf9-348f-4c12-b48e-c7092b816d46 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "DELETE /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124" status: 202 len: 0 microversion: 2.1 time: 0.175602 Jul 05 10:10:19.683047 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 20/40] 10.222.0.20 () {58 vars in 1247 bytes} [Tue Jul 5 10:10:19 2022] DELETE /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124 => generated 0 bytes in 177 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:10:19.691415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0619470c-2dba-4b89-8266-b6cbafd46364 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8a2fe040>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:19.805969 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0619470c-2dba-4b89-8266-b6cbafd46364 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124" status: 200 len: 345 microversion: 2.1 time: 0.118954 Jul 05 10:10:19.807033 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 21/41] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:19 2022] GET /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124 => generated 345 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:10:20.298784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-49caadb6-e266-4110-a265-b1ef23ec5650 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Action: 'create', calling method: .version_select at 0x7feb8b491040>, body: {"keypair": {"name": "tempest-keypair-1853656315", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG/Ohpg+OsT5gd6fdUTAb6u/+qyZz7OfkECmuUOZt4c/nKPacKKHFKYB1Y3G9jlR4MiKH4hgzwYgqMDkw141dLZPt6Qeei0aLqZifmORd3c0tqTERb1fZmdjJF0SG0yVsw=="}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:20.438437 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-49caadb6-e266-4110-a265-b1ef23ec5650 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:10:20.461642 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-49caadb6-e266-4110-a265-b1ef23ec5650 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for user 036cb887c7894e75ba3eb4c1a8b21c69 and project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:10:20.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-49caadb6-e266-4110-a265-b1ef23ec5650 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.066262 Jul 05 10:10:20.593191 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 21/42] 10.222.0.20 () {60 vars in 1158 bytes} [Tue Jul 5 10:10:19 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1068 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:20.593534 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6aa5e5bd-ebab-4573-8468-5fcf929839d0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:20.594149 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6aa5e5bd-ebab-4573-8468-5fcf929839d0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:20.613955 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6aa5e5bd-ebab-4573-8468-5fcf929839d0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:20.614824 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6aa5e5bd-ebab-4573-8468-5fcf929839d0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:20.617106 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6aa5e5bd-ebab-4573-8468-5fcf929839d0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:20.617592 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6aa5e5bd-ebab-4573-8468-5fcf929839d0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:20.725605 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1684848753", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:20.932682 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.network.neutron [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] validate_networks() for [('9dc39dd5-d322-430b-ba08-aae3d3ccc6d0', None, None, None, None, None)] {{(pid=117813) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:10:21.342648 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6aa5e5bd-ebab-4573-8468-5fcf929839d0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:21.367421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6aa5e5bd-ebab-4573-8468-5fcf929839d0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1273 microversion: 2.1 time: 0.777986 Jul 05 10:10:21.381487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 22/43] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:20 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1273 bytes in 793 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:21.398020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-29d7a512-e187-406c-bb76-dc3ca8c74ab4 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8ab939d0>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:21.450969 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-29d7a512-e187-406c-bb76-dc3ca8c74ab4 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] HTTP exception thrown: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-29d7a512-e187-406c-bb76-dc3ca8c74ab4 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Request method failure captured: Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124 HTTP/1.1 Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA361yXo1lbM2IMddZmvV505MNtBrbX23Zw-4dNEeqvOsreYY1jfHS0RSRhOEMdg7oIW-szddpKuBvkIvsX_df2UbGsOfvlPrAsZ8UZ8GY9SBEsoCt5ZjX8yCR4S5AeHedMPWPFSgGwjRj6gPvInXrHhSInOWqoposxqPKKNDo6Epbs Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: a75ede11bffb460aa6cb57e232cfa4d4 Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1214286970 Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:10:21.454740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:10:21.456070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/a75ede11bffb460aa6cb57e232cfa4d4"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_a75ede11bffb460aa6cb57e232cfa4d4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/a75ede11bffb460aa6cb57e232cfa4d4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/a75ede11bffb460aa6cb57e232cfa4d4"}]}] Jul 05 10:10:21.456070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1214286970 Jul 05 10:10:21.456070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: a75ede11bffb460aa6cb57e232cfa4d4 Jul 05 10:10:21.456070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1214286970 Jul 05 10:10:21.456070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-VolumesSnapshotsTestJSON-1214286970-project Jul 05 10:10:21.456070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:10:21.456070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:10:21.456070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: eca9bd479a194bfaabb8135605b8bf26 Jul 05 10:10:21.456070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1214286970-project Jul 05 10:10:21.456070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: Jul 05 10:10:21.456070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.456070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': '516dd2b4-ea0e-4117-91b0-ac7e38b19124'} Jul 05 10:10:21.456070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. (HTTP 404) (Request-ID: req-49c10a25-72cc-438c-9f11-3be4a2d07f71) Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:21.456888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 05 10:10:21.458020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 05 10:10:21.459187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:10:21.460280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.460280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.460280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:21.460280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.460280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:21.460280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:21.460280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:21.460280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:10:21.460280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:10:21.460280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.460280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.464152 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-29d7a512-e187-406c-bb76-dc3ca8c74ab4 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Returning 404 to user: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:10:21.465228 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-29d7a512-e187-406c-bb76-dc3ca8c74ab4 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124" status: 404 len: 109 microversion: 2.1 time: 0.082216 Jul 05 10:10:21.467593 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 23/44] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:21 2022] GET /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124 => generated 109 bytes in 85 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:10:21.486592 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a7124b8b-2cea-4184-90e4-15f77f64cdff tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8ab34ca0>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:21.569947 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-a7124b8b-2cea-4184-90e4-15f77f64cdff tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] HTTP exception thrown: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a7124b8b-2cea-4184-90e4-15f77f64cdff tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Request method failure captured: Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: DELETE /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124 HTTP/1.1 Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA361yXo1lbM2IMddZmvV505MNtBrbX23Zw-4dNEeqvOsreYY1jfHS0RSRhOEMdg7oIW-szddpKuBvkIvsX_df2UbGsOfvlPrAsZ8UZ8GY9SBEsoCt5ZjX8yCR4S5AeHedMPWPFSgGwjRj6gPvInXrHhSInOWqoposxqPKKNDo6Epbs Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: a75ede11bffb460aa6cb57e232cfa4d4 Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1214286970 Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:10:21.573060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:10:21.574379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/a75ede11bffb460aa6cb57e232cfa4d4"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_a75ede11bffb460aa6cb57e232cfa4d4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/a75ede11bffb460aa6cb57e232cfa4d4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/a75ede11bffb460aa6cb57e232cfa4d4"}]}] Jul 05 10:10:21.574379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1214286970 Jul 05 10:10:21.574379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: a75ede11bffb460aa6cb57e232cfa4d4 Jul 05 10:10:21.574379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1214286970 Jul 05 10:10:21.574379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-VolumesSnapshotsTestJSON-1214286970-project Jul 05 10:10:21.574379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:10:21.574379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:10:21.574379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: eca9bd479a194bfaabb8135605b8bf26 Jul 05 10:10:21.574379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1214286970-project Jul 05 10:10:21.574379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: Jul 05 10:10:21.574379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.574379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': '516dd2b4-ea0e-4117-91b0-ac7e38b19124'} Jul 05 10:10:21.574379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. (HTTP 404) (Request-ID: req-021d82ab-0b3b-454a-9723-3da81b84f51a) Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:21.575375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 05 10:10:21.576518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 05 10:10:21.577634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:10:21.579253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.579253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.579253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:21.579253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.579253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:21.579253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:21.579253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:21.579253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:10:21.579253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:10:21.579253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.579253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.579253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a7124b8b-2cea-4184-90e4-15f77f64cdff tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Returning 404 to user: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:10:21.581487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a7124b8b-2cea-4184-90e4-15f77f64cdff tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "DELETE /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124" status: 404 len: 109 microversion: 2.1 time: 0.102023 Jul 05 10:10:21.582130 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 24/45] 10.222.0.20 () {58 vars in 1247 bytes} [Tue Jul 5 10:10:21 2022] DELETE /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124 => generated 109 bytes in 109 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 05 10:10:21.601458 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-721461c0-2cbf-454e-8ae9-c43cd7e441f2 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Calling method '.version_select at 0x7feb8ab34c10>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:21.670713 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-721461c0-2cbf-454e-8ae9-c43cd7e441f2 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] HTTP exception thrown: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-721461c0-2cbf-454e-8ae9-c43cd7e441f2 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Request method failure captured: Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124 HTTP/1.1 Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA361yXo1lbM2IMddZmvV505MNtBrbX23Zw-4dNEeqvOsreYY1jfHS0RSRhOEMdg7oIW-szddpKuBvkIvsX_df2UbGsOfvlPrAsZ8UZ8GY9SBEsoCt5ZjX8yCR4S5AeHedMPWPFSgGwjRj6gPvInXrHhSInOWqoposxqPKKNDo6Epbs Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: a75ede11bffb460aa6cb57e232cfa4d4 Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1214286970 Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:10:21.673104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:10:21.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/a75ede11bffb460aa6cb57e232cfa4d4"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_a75ede11bffb460aa6cb57e232cfa4d4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/a75ede11bffb460aa6cb57e232cfa4d4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/a75ede11bffb460aa6cb57e232cfa4d4"}]}] Jul 05 10:10:21.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1214286970 Jul 05 10:10:21.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: a75ede11bffb460aa6cb57e232cfa4d4 Jul 05 10:10:21.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1214286970 Jul 05 10:10:21.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-VolumesSnapshotsTestJSON-1214286970-project Jul 05 10:10:21.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:10:21.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:10:21.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: eca9bd479a194bfaabb8135605b8bf26 Jul 05 10:10:21.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1214286970-project Jul 05 10:10:21.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: Jul 05 10:10:21.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': '516dd2b4-ea0e-4117-91b0-ac7e38b19124'} Jul 05 10:10:21.674470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. (HTTP 404) (Request-ID: req-fd031d29-b14b-48ad-9aca-4ebb7378012b) Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:21.675251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 05 10:10:21.677712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 05 10:10:21.680268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-721461c0-2cbf-454e-8ae9-c43cd7e441f2 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] Returning 404 to user: Volume 516dd2b4-ea0e-4117-91b0-ac7e38b19124 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-721461c0-2cbf-454e-8ae9-c43cd7e441f2 tempest-VolumesSnapshotsTestJSON-1214286970 tempest-VolumesSnapshotsTestJSON-1214286970-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124" status: 404 len: 109 microversion: 2.1 time: 0.087736 Jul 05 10:10:21.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 25/46] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:21 2022] GET /compute/v2.1/os-volumes/516dd2b4-ea0e-4117-91b0-ac7e38b19124 => generated 109 bytes in 89 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 05 10:10:22.387632 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-442f88d2-37bc-4d7c-8ed2-a6c3cab0a08e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:22.388463 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-442f88d2-37bc-4d7c-8ed2-a6c3cab0a08e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:22.405962 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-442f88d2-37bc-4d7c-8ed2-a6c3cab0a08e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:22.406771 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-442f88d2-37bc-4d7c-8ed2-a6c3cab0a08e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:22.407772 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-442f88d2-37bc-4d7c-8ed2-a6c3cab0a08e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:22.408497 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-442f88d2-37bc-4d7c-8ed2-a6c3cab0a08e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:22.813720 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-442f88d2-37bc-4d7c-8ed2-a6c3cab0a08e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:22.848579 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-442f88d2-37bc-4d7c-8ed2-a6c3cab0a08e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1273 microversion: 2.1 time: 0.453813 Jul 05 10:10:22.849796 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 26/47] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:22 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1273 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:23.202135 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Flavor limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:10:23.203418 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Image limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:10:23.204121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Flavor pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:10:23.207901 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Image pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:10:23.209813 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:10:23.228721 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:23.229579 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:23.232852 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:23.235086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:23.279489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Getting quotas for project 3f2199fa0bc54901aa24195a250a9e25. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:10:23.285975 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Getting quotas for user 248946c73cb245f9b8e6fe73e46133cf and project 3f2199fa0bc54901aa24195a250a9e25. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:10:23.337295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Going to run 1 instances... {{(pid=117813) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:10:23.347487 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117813) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:10:23.473417 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d425a553-2432-440f-bc3a-dd6d25c5df86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.971443 Jul 05 10:10:23.478399 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 22/48] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:10:20 2022] POST /compute/v2.1/servers => generated 383 bytes in 2977 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 05 10:10:23.488066 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-819323c8-4553-4541-acfb-985e9c850687 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:23.490566 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-819323c8-4553-4541-acfb-985e9c850687 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:23.778793 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-819323c8-4553-4541-acfb-985e9c850687 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:23.787786 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-819323c8-4553-4541-acfb-985e9c850687 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1222 microversion: 2.1 time: 0.303897 Jul 05 10:10:23.792430 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 27/49] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:23 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1222 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:23.866126 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-cd6cda86-de6f-486a-8c8e-7c540b0479cf tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:23.867077 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-cd6cda86-de6f-486a-8c8e-7c540b0479cf tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:23.882225 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cd6cda86-de6f-486a-8c8e-7c540b0479cf tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:23.883377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cd6cda86-de6f-486a-8c8e-7c540b0479cf tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:23.885783 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cd6cda86-de6f-486a-8c8e-7c540b0479cf tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:23.887659 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cd6cda86-de6f-486a-8c8e-7c540b0479cf tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:24.172560 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1407824980", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:24.440820 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.network.neutron [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] validate_networks() for [('9dc39dd5-d322-430b-ba08-aae3d3ccc6d0', None, None, None, None, None)] {{(pid=117812) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:10:24.491505 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-cd6cda86-de6f-486a-8c8e-7c540b0479cf tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:24.516937 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-cd6cda86-de6f-486a-8c8e-7c540b0479cf tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.654078 Jul 05 10:10:24.520717 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 23/50] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:23 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 660 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:25.539314 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ea448073-50b3-47f8-9ebe-5b41f89025a3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:25.541388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ea448073-50b3-47f8-9ebe-5b41f89025a3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:25.554885 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ea448073-50b3-47f8-9ebe-5b41f89025a3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:25.555342 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ea448073-50b3-47f8-9ebe-5b41f89025a3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:25.557844 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ea448073-50b3-47f8-9ebe-5b41f89025a3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:25.571743 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ea448073-50b3-47f8-9ebe-5b41f89025a3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:26.696120 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Flavor limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:10:26.703486 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Image limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:10:26.704113 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Flavor pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:10:26.704577 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Image pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:10:26.705018 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:10:26.749652 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ea448073-50b3-47f8-9ebe-5b41f89025a3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:26.752514 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:26.753456 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:26.754620 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:26.756529 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:26.758629 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ea448073-50b3-47f8-9ebe-5b41f89025a3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 1.225262 Jul 05 10:10:26.763260 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 24/51] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:25 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 1231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:26.824641 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Getting quotas for project 3f2199fa0bc54901aa24195a250a9e25. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:10:26.831878 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Getting quotas for user 248946c73cb245f9b8e6fe73e46133cf and project 3f2199fa0bc54901aa24195a250a9e25. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:10:26.864161 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Going to run 1 instances... {{(pid=117812) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:10:26.870159 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117812) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:10:27.027443 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-1afb8138-3f20-4bf2-b9ea-359dd88360db tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.861838 Jul 05 10:10:27.034415 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 28/52] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:10:24 2022] POST /compute/v2.1/servers => generated 383 bytes in 2864 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:10:27.040008 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-463531879", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "033e3c55-aad3-40e5-8b8b-1dedb7b873c7"}], "security_groups": [{"name": "tempest-securitygroup--186075987"}], "key_name": "tempest-keypair-1853656315", "user_data": "}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:27.497888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-65a1e781-928e-4ae1-8e9c-ff094db70897 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Action: 'create', calling method: .version_select at 0x7feb8a2bd940>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1614160155", "metadata": {"Type": "work"}}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-65a1e781-928e-4ae1-8e9c-ff094db70897 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Request method failure captured: Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 122 Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4SeAxrWU03L8hqC7RW6AgWZ_DJUeP32Bp2gw3TzRwnF0Fe2Qha11qWqVGf7cSSsQJEPhruFZYvLQKbiFPJvm8zkE-GfsJSQU_N8_vHxd_b1I5sVFJHssdSuthp5epy8DQLF4i0760DbunfBxIo5D-zzCabM35eQgioNxr5h-TsAc0 Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: fe78696ea79041729421265731148164 Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:10:27.545065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:10:27.547078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/fe78696ea79041729421265731148164"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_fe78696ea79041729421265731148164"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/fe78696ea79041729421265731148164"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/fe78696ea79041729421265731148164"}]}] Jul 05 10:10:27.547078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:27.547078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: fe78696ea79041729421265731148164 Jul 05 10:10:27.547078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:27.547078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-VolumesNegativeTest-491224343-project Jul 05 10:10:27.547078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:10:27.547078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:10:27.547078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 6512b355e1ce4824ad69eb9e23cbea95 Jul 05 10:10:27.547078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-VolumesNegativeTest-491224343-project Jul 05 10:10:27.547078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Jul 05 10:10:27.547078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1614160155", "metadata": {"Type": "work"}}} Jul 05 10:10:27.547078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: Jul 05 10:10:27.547078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1614160155', 'metadata': {'Type': 'work'}}}} Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise error Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi '#$%' Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:10:27.548294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 05 10:10:27.553741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 05 10:10:27.557817 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 05 10:10:27.557817 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:27.557817 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-65a1e781-928e-4ae1-8e9c-ff094db70897 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:10:27.572118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-65a1e781-928e-4ae1-8e9c-ff094db70897 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] 10.222.0.20 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.806394 Jul 05 10:10:27.579796 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 25/53] 10.222.0.20 () {60 vars in 1155 bytes} [Tue Jul 5 10:10:26 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 816 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 05 10:10:27.611340 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ed99be51-0689-4c83-ad8a-8433b9f88b51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:27.626971 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ed99be51-0689-4c83-ad8a-8433b9f88b51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:27.728725 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.network.neutron [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] validate_networks() for [('033e3c55-aad3-40e5-8b8b-1dedb7b873c7', None, None, None, None, None)] {{(pid=117812) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:10:28.271141 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ed99be51-0689-4c83-ad8a-8433b9f88b51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:28.280044 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ed99be51-0689-4c83-ad8a-8433b9f88b51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1222 microversion: 2.1 time: 0.696083 Jul 05 10:10:28.285392 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 26/54] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:27 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1222 bytes in 705 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:28.306817 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-23799ea8-3d1c-4e99-80e7-c5bfa969fcb6 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Action: 'create', calling method: .version_select at 0x7feb8abb1dc0>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-381684314", "metadata": {"Type": "work"}}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-23799ea8-3d1c-4e99-80e7-c5bfa969fcb6 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Request method failure captured: Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 119 Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4SeAxrWU03L8hqC7RW6AgWZ_DJUeP32Bp2gw3TzRwnF0Fe2Qha11qWqVGf7cSSsQJEPhruFZYvLQKbiFPJvm8zkE-GfsJSQU_N8_vHxd_b1I5sVFJHssdSuthp5epy8DQLF4i0760DbunfBxIo5D-zzCabM35eQgioNxr5h-TsAc0 Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: fe78696ea79041729421265731148164 Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:10:28.309989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:10:28.311121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/fe78696ea79041729421265731148164"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_fe78696ea79041729421265731148164"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/fe78696ea79041729421265731148164"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/fe78696ea79041729421265731148164"}]}] Jul 05 10:10:28.311121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:28.311121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: fe78696ea79041729421265731148164 Jul 05 10:10:28.311121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:28.311121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-VolumesNegativeTest-491224343-project Jul 05 10:10:28.311121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:10:28.311121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:10:28.311121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 6512b355e1ce4824ad69eb9e23cbea95 Jul 05 10:10:28.311121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-VolumesNegativeTest-491224343-project Jul 05 10:10:28.311121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Jul 05 10:10:28.311121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-381684314", "metadata": {"Type": "work"}}} Jul 05 10:10:28.311121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: Jul 05 10:10:28.311121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-381684314', 'metadata': {'Type': 'work'}}}} Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise error Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi '0' Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:10:28.311834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 05 10:10:28.313004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 05 10:10:28.313965 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 05 10:10:28.313965 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:28.315667 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-23799ea8-3d1c-4e99-80e7-c5bfa969fcb6 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:10:28.316702 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-23799ea8-3d1c-4e99-80e7-c5bfa969fcb6 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] 10.222.0.20 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.030075 Jul 05 10:10:28.321321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 27/55] 10.222.0.20 () {60 vars in 1155 bytes} [Tue Jul 5 10:10:28 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 35 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 05 10:10:28.345032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-986e5aaa-2b6f-4073-ab3d-649adeb6011b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:28.345652 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-986e5aaa-2b6f-4073-ab3d-649adeb6011b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:28.373612 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-986e5aaa-2b6f-4073-ab3d-649adeb6011b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:28.374079 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-986e5aaa-2b6f-4073-ab3d-649adeb6011b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:28.374870 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-986e5aaa-2b6f-4073-ab3d-649adeb6011b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:28.375310 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-986e5aaa-2b6f-4073-ab3d-649adeb6011b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:29.022444 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-986e5aaa-2b6f-4073-ab3d-649adeb6011b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:29.036144 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-986e5aaa-2b6f-4073-ab3d-649adeb6011b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.698395 Jul 05 10:10:29.044148 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 28/56] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:28 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 708 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:29.063680 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f708935c-cb97-4179-859d-b5e084f976f7 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Action: 'create', calling method: .version_select at 0x7feb8a2d8310>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1198438558", "metadata": {"Type": "work"}}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f708935c-cb97-4179-859d-b5e084f976f7 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Request method failure captured: Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 119 Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4SeAxrWU03L8hqC7RW6AgWZ_DJUeP32Bp2gw3TzRwnF0Fe2Qha11qWqVGf7cSSsQJEPhruFZYvLQKbiFPJvm8zkE-GfsJSQU_N8_vHxd_b1I5sVFJHssdSuthp5epy8DQLF4i0760DbunfBxIo5D-zzCabM35eQgioNxr5h-TsAc0 Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: fe78696ea79041729421265731148164 Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:10:29.077086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:10:29.078407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/fe78696ea79041729421265731148164"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_fe78696ea79041729421265731148164"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/fe78696ea79041729421265731148164"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/fe78696ea79041729421265731148164"}]}] Jul 05 10:10:29.078407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:29.078407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: fe78696ea79041729421265731148164 Jul 05 10:10:29.078407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:29.078407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-VolumesNegativeTest-491224343-project Jul 05 10:10:29.078407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:10:29.078407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:10:29.078407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 6512b355e1ce4824ad69eb9e23cbea95 Jul 05 10:10:29.078407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-VolumesNegativeTest-491224343-project Jul 05 10:10:29.078407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Jul 05 10:10:29.078407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1198438558", "metadata": {"Type": "work"}}} Jul 05 10:10:29.078407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: Jul 05 10:10:29.078407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1198438558', 'metadata': {'Type': 'work'}}}} Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise error Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi '' Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:10:29.079214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 05 10:10:29.080427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 05 10:10:29.081494 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 05 10:10:29.081494 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.099212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f708935c-cb97-4179-859d-b5e084f976f7 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:10:29.100497 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f708935c-cb97-4179-859d-b5e084f976f7 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] 10.222.0.20 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.054860 Jul 05 10:10:29.105348 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 29/57] 10.222.0.20 () {60 vars in 1155 bytes} [Tue Jul 5 10:10:29 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 60 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 05 10:10:29.112204 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1a8bd415-102f-4019-afe7-9d6411cf0fb7 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Calling method '.version_select at 0x7feb8a2bd940>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:29.200420 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-1a8bd415-102f-4019-afe7-9d6411cf0fb7 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] HTTP exception thrown: Volume tempest-invalid-1577744037 could not be found. Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1a8bd415-102f-4019-afe7-9d6411cf0fb7 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Request method failure captured: Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1577744037 HTTP/1.1 Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4SeAxrWU03L8hqC7RW6AgWZ_DJUeP32Bp2gw3TzRwnF0Fe2Qha11qWqVGf7cSSsQJEPhruFZYvLQKbiFPJvm8zkE-GfsJSQU_N8_vHxd_b1I5sVFJHssdSuthp5epy8DQLF4i0760DbunfBxIo5D-zzCabM35eQgioNxr5h-TsAc0 Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: fe78696ea79041729421265731148164 Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:10:29.204921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:10:29.206343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/fe78696ea79041729421265731148164"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_fe78696ea79041729421265731148164"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/fe78696ea79041729421265731148164"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/fe78696ea79041729421265731148164"}]}] Jul 05 10:10:29.206343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:29.206343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: fe78696ea79041729421265731148164 Jul 05 10:10:29.206343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:29.206343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-VolumesNegativeTest-491224343-project Jul 05 10:10:29.206343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:10:29.206343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:10:29.206343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 6512b355e1ce4824ad69eb9e23cbea95 Jul 05 10:10:29.206343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-VolumesNegativeTest-491224343-project Jul 05 10:10:29.206343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: Jul 05 10:10:29.206343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Volume tempest-invalid-1577744037 could not be found. Jul 05 10:10:29.206343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'tempest-invalid-1577744037'} Jul 05 10:10:29.206343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1577744037 could not be found. (HTTP 404) (Request-ID: req-d2cd138e-1a56-4224-94fd-6424e6fcaee3) Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:29.207053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 05 10:10:29.208201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1577744037 could not be found. Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:10:29.209289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1577744037 could not be found. Jul 05 10:10:29.210482 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.210482 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:29.210482 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.210482 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:29.210482 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:29.210482 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:29.210482 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:10:29.210482 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:10:29.210482 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1577744037 could not be found. Jul 05 10:10:29.210482 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.212076 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1a8bd415-102f-4019-afe7-9d6411cf0fb7 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Returning 404 to user: Volume tempest-invalid-1577744037 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:10:29.213891 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1a8bd415-102f-4019-afe7-9d6411cf0fb7 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] 10.222.0.20 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1577744037" status: 404 len: 99 microversion: 2.1 time: 0.106763 Jul 05 10:10:29.214585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 30/58] 10.222.0.20 () {58 vars in 1217 bytes} [Tue Jul 5 10:10:29 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1577744037 => generated 99 bytes in 109 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 05 10:10:29.247728 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-894614fb-4b92-4385-8d77-4ca30ba12d9f tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] 10.222.0.20 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.011029 Jul 05 10:10:29.251776 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 31/59] 10.222.0.20 () {58 vars in 1139 bytes} [Tue Jul 5 10:10:29 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 31 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 05 10:10:29.274271 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e01d18fa-5249-4c94-801c-a3f9a0349d3d tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.010698 Jul 05 10:10:29.278567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 32/60] 10.222.0.20 () {58 vars in 1136 bytes} [Tue Jul 5 10:10:29 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 15 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 05 10:10:29.297791 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9eb4a037-701d-466d-afea-62af964201c6 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Calling method '.version_select at 0x7feb8a2d85e0>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:29.375229 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-9eb4a037-701d-466d-afea-62af964201c6 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] HTTP exception thrown: Volume 8add7471-d82b-4d1c-92dd-0a8635ecc85f could not be found. Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9eb4a037-701d-466d-afea-62af964201c6 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Request method failure captured: Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: DELETE /compute/v2.1/os-volumes/8add7471-d82b-4d1c-92dd-0a8635ecc85f HTTP/1.1 Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4SeAxrWU03L8hqC7RW6AgWZ_DJUeP32Bp2gw3TzRwnF0Fe2Qha11qWqVGf7cSSsQJEPhruFZYvLQKbiFPJvm8zkE-GfsJSQU_N8_vHxd_b1I5sVFJHssdSuthp5epy8DQLF4i0760DbunfBxIo5D-zzCabM35eQgioNxr5h-TsAc0 Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: fe78696ea79041729421265731148164 Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:10:29.378658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:10:29.379925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/fe78696ea79041729421265731148164"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_fe78696ea79041729421265731148164"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/fe78696ea79041729421265731148164"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/fe78696ea79041729421265731148164"}]}] Jul 05 10:10:29.379925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:29.379925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: fe78696ea79041729421265731148164 Jul 05 10:10:29.379925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:29.379925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-VolumesNegativeTest-491224343-project Jul 05 10:10:29.379925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:10:29.379925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:10:29.379925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 6512b355e1ce4824ad69eb9e23cbea95 Jul 05 10:10:29.379925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-VolumesNegativeTest-491224343-project Jul 05 10:10:29.379925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: Jul 05 10:10:29.379925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Volume 8add7471-d82b-4d1c-92dd-0a8635ecc85f could not be found. Jul 05 10:10:29.379925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': '8add7471-d82b-4d1c-92dd-0a8635ecc85f'} Jul 05 10:10:29.379925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 8add7471-d82b-4d1c-92dd-0a8635ecc85f could not be found. (HTTP 404) (Request-ID: req-f0c94123-3b59-44eb-a14e-21169033144b) Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:29.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 05 10:10:29.382019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 8add7471-d82b-4d1c-92dd-0a8635ecc85f could not be found. Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 05 10:10:29.383149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:10:29.384287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 8add7471-d82b-4d1c-92dd-0a8635ecc85f could not be found. Jul 05 10:10:29.384287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.384287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:29.384287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.384287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:29.384287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:29.384287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:29.384287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:10:29.384287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:10:29.384287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 8add7471-d82b-4d1c-92dd-0a8635ecc85f could not be found. Jul 05 10:10:29.384287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:29.387031 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9eb4a037-701d-466d-afea-62af964201c6 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Returning 404 to user: Volume 8add7471-d82b-4d1c-92dd-0a8635ecc85f could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:10:29.388717 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9eb4a037-701d-466d-afea-62af964201c6 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] 10.222.0.20 "DELETE /compute/v2.1/os-volumes/8add7471-d82b-4d1c-92dd-0a8635ecc85f" status: 404 len: 109 microversion: 2.1 time: 0.097090 Jul 05 10:10:29.389716 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 33/61] 10.222.0.20 () {58 vars in 1247 bytes} [Tue Jul 5 10:10:29 2022] DELETE /compute/v2.1/os-volumes/8add7471-d82b-4d1c-92dd-0a8635ecc85f => generated 109 bytes in 99 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:10:29.405176 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9f2a6101-7aa0-436d-addb-140c42691006 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:29.409161 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9f2a6101-7aa0-436d-addb-140c42691006 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:29.439750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9f2a6101-7aa0-436d-addb-140c42691006 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:29.441809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9f2a6101-7aa0-436d-addb-140c42691006 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:29.449563 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9f2a6101-7aa0-436d-addb-140c42691006 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:29.453408 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9f2a6101-7aa0-436d-addb-140c42691006 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:30.287874 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9f2a6101-7aa0-436d-addb-140c42691006 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:30.315224 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9f2a6101-7aa0-436d-addb-140c42691006 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1274 microversion: 2.1 time: 0.923629 Jul 05 10:10:30.316178 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 34/62] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:29 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1274 bytes in 926 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:30.337514 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-69cb9602-5812-444b-bc2d-8c7ea5fb7583 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Calling method '.version_select at 0x7feb8a2d83a0>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:30.388436 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-69cb9602-5812-444b-bc2d-8c7ea5fb7583 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] HTTP exception thrown: Volume cd92e346-1494-489c-b105-530352718271 could not be found. Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-69cb9602-5812-444b-bc2d-8c7ea5fb7583 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Request method failure captured: Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/os-volumes/cd92e346-1494-489c-b105-530352718271 HTTP/1.1 Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4SeAxrWU03L8hqC7RW6AgWZ_DJUeP32Bp2gw3TzRwnF0Fe2Qha11qWqVGf7cSSsQJEPhruFZYvLQKbiFPJvm8zkE-GfsJSQU_N8_vHxd_b1I5sVFJHssdSuthp5epy8DQLF4i0760DbunfBxIo5D-zzCabM35eQgioNxr5h-TsAc0 Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: fe78696ea79041729421265731148164 Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:10:30.391569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:10:30.392839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/fe78696ea79041729421265731148164"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_fe78696ea79041729421265731148164"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/fe78696ea79041729421265731148164"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/fe78696ea79041729421265731148164"}]}] Jul 05 10:10:30.392839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:30.392839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: fe78696ea79041729421265731148164 Jul 05 10:10:30.392839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-VolumesNegativeTest-491224343 Jul 05 10:10:30.392839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-VolumesNegativeTest-491224343-project Jul 05 10:10:30.392839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:10:30.392839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:10:30.392839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 6512b355e1ce4824ad69eb9e23cbea95 Jul 05 10:10:30.392839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-VolumesNegativeTest-491224343-project Jul 05 10:10:30.392839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: Jul 05 10:10:30.392839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Volume cd92e346-1494-489c-b105-530352718271 could not be found. Jul 05 10:10:30.392839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'cd92e346-1494-489c-b105-530352718271'} Jul 05 10:10:30.392839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume cd92e346-1494-489c-b105-530352718271 could not be found. (HTTP 404) (Request-ID: req-f28bc873-0111-4897-87a2-253585e41887) Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:30.393633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 05 10:10:30.394972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume cd92e346-1494-489c-b105-530352718271 could not be found. Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 05 10:10:30.396321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:10:30.397688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume cd92e346-1494-489c-b105-530352718271 could not be found. Jul 05 10:10:30.397688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:30.397688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:10:30.397688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:30.397688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:10:30.397688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:10:30.397688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:10:30.397688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:10:30.397688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:10:30.397688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume cd92e346-1494-489c-b105-530352718271 could not be found. Jul 05 10:10:30.397688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:10:30.400052 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-69cb9602-5812-444b-bc2d-8c7ea5fb7583 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] Returning 404 to user: Volume cd92e346-1494-489c-b105-530352718271 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:10:30.400052 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-69cb9602-5812-444b-bc2d-8c7ea5fb7583 tempest-VolumesNegativeTest-491224343 tempest-VolumesNegativeTest-491224343-project] 10.222.0.20 "GET /compute/v2.1/os-volumes/cd92e346-1494-489c-b105-530352718271" status: 404 len: 109 microversion: 2.1 time: 0.077859 Jul 05 10:10:30.400544 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 35/63] 10.222.0.20 () {58 vars in 1244 bytes} [Tue Jul 5 10:10:30 2022] GET /compute/v2.1/os-volumes/cd92e346-1494-489c-b105-530352718271 => generated 109 bytes in 82 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:10:30.426653 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d1a66e7b-ee0a-41df-8064-93ecbb8f2131 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:30.434091 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d1a66e7b-ee0a-41df-8064-93ecbb8f2131 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:30.482243 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d1a66e7b-ee0a-41df-8064-93ecbb8f2131 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:30.485948 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d1a66e7b-ee0a-41df-8064-93ecbb8f2131 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:30.493839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d1a66e7b-ee0a-41df-8064-93ecbb8f2131 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:30.497409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d1a66e7b-ee0a-41df-8064-93ecbb8f2131 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:30.960839 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Flavor limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:10:30.961417 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Image limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:10:30.961705 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Flavor pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:10:30.962317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Image pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:10:30.962876 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:10:30.999616 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:31.000076 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:31.000951 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:31.001373 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:31.100861 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:10:31.117854 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for user 036cb887c7894e75ba3eb4c1a8b21c69 and project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:10:31.166963 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Going to run 1 instances... {{(pid=117812) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:10:31.180136 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117812) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:10:31.382742 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e4406690-ac0d-4076-8069-935b9903c5de tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 4.352589 Jul 05 10:10:31.383540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 29/64] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:10:27 2022] POST /compute/v2.1/servers => generated 401 bytes in 4354 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:10:31.394662 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-96f49135-bbc2-4337-88f4-a06e48a55401 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:31.396559 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-96f49135-bbc2-4337-88f4-a06e48a55401 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:31.412881 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-96f49135-bbc2-4337-88f4-a06e48a55401 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:31.413439 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-96f49135-bbc2-4337-88f4-a06e48a55401 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:31.414425 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-96f49135-bbc2-4337-88f4-a06e48a55401 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:31.414976 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-96f49135-bbc2-4337-88f4-a06e48a55401 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:31.833141 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-96f49135-bbc2-4337-88f4-a06e48a55401 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:31.862109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-96f49135-bbc2-4337-88f4-a06e48a55401 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1280 microversion: 2.1 time: 0.461250 Jul 05 10:10:31.866607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 30/65] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:31 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1280 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:31.875948 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3e0ad428-b498-4c56-8e39-d8f771479157 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:31.876757 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3e0ad428-b498-4c56-8e39-d8f771479157 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:32.328947 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d1a66e7b-ee0a-41df-8064-93ecbb8f2131 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:32.342274 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d1a66e7b-ee0a-41df-8064-93ecbb8f2131 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 1.940212 Jul 05 10:10:32.354413 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 36/66] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:30 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 1953 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:32.644810 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-3e0ad428-b498-4c56-8e39-d8f771479157 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:32.654780 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3e0ad428-b498-4c56-8e39-d8f771479157 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1241 microversion: 2.1 time: 0.786048 Jul 05 10:10:32.656928 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 31/67] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:31 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1241 bytes in 789 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:32.794210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-0cd901f5-1430-4543-8485-0ee0e7079dda req-b4002279-a7cd-4973-9d40-0295aa27f06e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b7ce1e24-0c58-4071-9610-57d072131857", "tag": "1590f070-4b3b-4762-9ba6-291ca1720666"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:32.820490 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-0cd901f5-1430-4543-8485-0ee0e7079dda req-b4002279-a7cd-4973-9d40-0295aa27f06e service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:32.823576 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-0cd901f5-1430-4543-8485-0ee0e7079dda req-b4002279-a7cd-4973-9d40-0295aa27f06e service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:32.883563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e4a47d21-9a81-4c01-a48c-5a450155f9b9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:32.884271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e4a47d21-9a81-4c01-a48c-5a450155f9b9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:32.896631 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-0cd901f5-1430-4543-8485-0ee0e7079dda req-b4002279-a7cd-4973-9d40-0295aa27f06e service nova] Creating event network-changed:1590f070-4b3b-4762-9ba6-291ca1720666 for instance b7ce1e24-0c58-4071-9610-57d072131857 on hv-cinder-maste Jul 05 10:10:32.899802 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-0cd901f5-1430-4543-8485-0ee0e7079dda req-b4002279-a7cd-4973-9d40-0295aa27f06e service nova] Acquired lock "compute-rpcapi-router" {{(pid=117813) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 05 10:10:32.902039 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d5c8bd0c-6c70-4ae9-8973-5de62fe94e46 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:32.903922 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e4a47d21-9a81-4c01-a48c-5a450155f9b9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:32.905163 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d5c8bd0c-6c70-4ae9-8973-5de62fe94e46 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:32.907022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e4a47d21-9a81-4c01-a48c-5a450155f9b9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:32.908080 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d5c8bd0c-6c70-4ae9-8973-5de62fe94e46 None None] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:32.909587 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d5c8bd0c-6c70-4ae9-8973-5de62fe94e46 None None] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:32.910965 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e4a47d21-9a81-4c01-a48c-5a450155f9b9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:32.912970 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e4a47d21-9a81-4c01-a48c-5a450155f9b9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:32.940243 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.compute.rpcapi [None req-d5c8bd0c-6c70-4ae9-8973-5de62fe94e46 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 05 10:10:32.943084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d5c8bd0c-6c70-4ae9-8973-5de62fe94e46 None None] Releasing lock "compute-rpcapi-router" {{(pid=117813) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 05 10:10:32.986177 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d5c8bd0c-6c70-4ae9-8973-5de62fe94e46 None None] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.351024 Jul 05 10:10:32.990923 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 37/68] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:10:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:10:33.151277 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e4a47d21-9a81-4c01-a48c-5a450155f9b9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:33.163771 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e4a47d21-9a81-4c01-a48c-5a450155f9b9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1280 microversion: 2.1 time: 0.284050 Jul 05 10:10:33.165032 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 32/69] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:32 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1280 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:33.361561 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-7f38fb0f-1b04-4643-9dcd-11814bf62db0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:33.365368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7f38fb0f-1b04-4643-9dcd-11814bf62db0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:33.378699 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7f38fb0f-1b04-4643-9dcd-11814bf62db0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:33.379155 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7f38fb0f-1b04-4643-9dcd-11814bf62db0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:33.385870 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7f38fb0f-1b04-4643-9dcd-11814bf62db0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:33.386563 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7f38fb0f-1b04-4643-9dcd-11814bf62db0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:33.673193 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-589af56d-4d0e-4af7-a193-84d6ba43a52d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:33.674427 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-589af56d-4d0e-4af7-a193-84d6ba43a52d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:33.686545 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-589af56d-4d0e-4af7-a193-84d6ba43a52d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:33.687137 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-589af56d-4d0e-4af7-a193-84d6ba43a52d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:33.689120 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-589af56d-4d0e-4af7-a193-84d6ba43a52d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:33.690081 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-589af56d-4d0e-4af7-a193-84d6ba43a52d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:33.875998 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-589af56d-4d0e-4af7-a193-84d6ba43a52d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:33.892774 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-589af56d-4d0e-4af7-a193-84d6ba43a52d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1293 microversion: 2.1 time: 0.223271 Jul 05 10:10:33.893361 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 33/70] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:33 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1293 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:34.171524 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-7f38fb0f-1b04-4643-9dcd-11814bf62db0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:34.181875 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-eb2e30d6-df71-4aaa-a65f-f6be59cba8b0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:34.182727 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-eb2e30d6-df71-4aaa-a65f-f6be59cba8b0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:34.187859 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-7f38fb0f-1b04-4643-9dcd-11814bf62db0 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.832199 Jul 05 10:10:34.188446 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 38/71] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:33 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 834 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:34.196097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eb2e30d6-df71-4aaa-a65f-f6be59cba8b0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:34.196832 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eb2e30d6-df71-4aaa-a65f-f6be59cba8b0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:34.197617 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eb2e30d6-df71-4aaa-a65f-f6be59cba8b0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:34.198058 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eb2e30d6-df71-4aaa-a65f-f6be59cba8b0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:34.911251 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-902ec6c7-1951-42b9-a762-8be75be47cda tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:34.911910 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-902ec6c7-1951-42b9-a762-8be75be47cda tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:34.939653 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-902ec6c7-1951-42b9-a762-8be75be47cda tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:34.940322 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-902ec6c7-1951-42b9-a762-8be75be47cda tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:34.943949 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-902ec6c7-1951-42b9-a762-8be75be47cda tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:34.945392 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-902ec6c7-1951-42b9-a762-8be75be47cda tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:35.003329 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-eb2e30d6-df71-4aaa-a65f-f6be59cba8b0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:35.013081 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-eb2e30d6-df71-4aaa-a65f-f6be59cba8b0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.834950 Jul 05 10:10:35.019077 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 34/72] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:34 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 842 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:35.207311 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6cdfc463-d593-419a-b422-fef45696127c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:35.208432 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6cdfc463-d593-419a-b422-fef45696127c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:35.219681 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6cdfc463-d593-419a-b422-fef45696127c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:35.220381 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6cdfc463-d593-419a-b422-fef45696127c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:35.221351 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6cdfc463-d593-419a-b422-fef45696127c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:35.221941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6cdfc463-d593-419a-b422-fef45696127c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:35.350981 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-902ec6c7-1951-42b9-a762-8be75be47cda tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:35.370115 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-902ec6c7-1951-42b9-a762-8be75be47cda tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1299 microversion: 2.1 time: 0.460951 Jul 05 10:10:35.384824 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 39/73] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:34 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1299 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:36.040378 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e0001fe1-104b-4af2-a048-57ae14cbb296 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:36.042382 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e0001fe1-104b-4af2-a048-57ae14cbb296 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:36.056042 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e0001fe1-104b-4af2-a048-57ae14cbb296 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:36.060112 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e0001fe1-104b-4af2-a048-57ae14cbb296 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:36.062430 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e0001fe1-104b-4af2-a048-57ae14cbb296 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:36.064685 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e0001fe1-104b-4af2-a048-57ae14cbb296 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:36.647423 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6cdfc463-d593-419a-b422-fef45696127c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:36.674071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6cdfc463-d593-419a-b422-fef45696127c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 1.470665 Jul 05 10:10:36.675429 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 35/74] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:35 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 1473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:36.688031 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d44f3878-f977-4e5d-b8b2-aa68182317c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:36.689751 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d44f3878-f977-4e5d-b8b2-aa68182317c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:36.724296 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d44f3878-f977-4e5d-b8b2-aa68182317c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:36.725056 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d44f3878-f977-4e5d-b8b2-aa68182317c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:36.725895 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d44f3878-f977-4e5d-b8b2-aa68182317c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:36.726339 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d44f3878-f977-4e5d-b8b2-aa68182317c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:36.905436 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d44f3878-f977-4e5d-b8b2-aa68182317c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:36.916985 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d44f3878-f977-4e5d-b8b2-aa68182317c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1299 microversion: 2.1 time: 0.240257 Jul 05 10:10:36.926704 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 36/75] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:36 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1299 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:37.038379 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e0001fe1-104b-4af2-a048-57ae14cbb296 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:37.047103 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e0001fe1-104b-4af2-a048-57ae14cbb296 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 1.011647 Jul 05 10:10:37.053060 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 40/76] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:36 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 1018 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:37.266918 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-c6f60046-0c88-4323-a0ea-d33588e5de3e req-a6497eb5-fa13-4be7-aba4-fb62dd49d56d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fc065612-7f7d-45fb-89cc-9f332fb15dd1", "tag": "ff58dc45-3b6a-4ad9-a225-3ec518174c19"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:37.296555 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-c6f60046-0c88-4323-a0ea-d33588e5de3e req-a6497eb5-fa13-4be7-aba4-fb62dd49d56d service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:37.297327 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-c6f60046-0c88-4323-a0ea-d33588e5de3e req-a6497eb5-fa13-4be7-aba4-fb62dd49d56d service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:37.358932 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-c6f60046-0c88-4323-a0ea-d33588e5de3e req-a6497eb5-fa13-4be7-aba4-fb62dd49d56d service nova] Creating event network-changed:ff58dc45-3b6a-4ad9-a225-3ec518174c19 for instance fc065612-7f7d-45fb-89cc-9f332fb15dd1 on hv-cinder-maste Jul 05 10:10:37.361590 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-c6f60046-0c88-4323-a0ea-d33588e5de3e req-a6497eb5-fa13-4be7-aba4-fb62dd49d56d service nova] Acquired lock "compute-rpcapi-router" {{(pid=117812) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 05 10:10:37.366837 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-60fc9af4-cca7-4250-98c1-45a2e505bfc0 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:37.369041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-60fc9af4-cca7-4250-98c1-45a2e505bfc0 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:37.370269 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-60fc9af4-cca7-4250-98c1-45a2e505bfc0 None None] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:37.378974 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-60fc9af4-cca7-4250-98c1-45a2e505bfc0 None None] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:37.408510 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.compute.rpcapi [None req-60fc9af4-cca7-4250-98c1-45a2e505bfc0 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 05 10:10:37.418086 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-60fc9af4-cca7-4250-98c1-45a2e505bfc0 None None] Releasing lock "compute-rpcapi-router" {{(pid=117812) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 05 10:10:37.464890 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-60fc9af4-cca7-4250-98c1-45a2e505bfc0 None None] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.203480 Jul 05 10:10:37.465894 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 37/77] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:10:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:37.691291 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b93656d9-16a8-4b4a-93e5-e028b6dd6a4e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:37.692071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b93656d9-16a8-4b4a-93e5-e028b6dd6a4e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:37.703881 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b93656d9-16a8-4b4a-93e5-e028b6dd6a4e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:37.704338 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b93656d9-16a8-4b4a-93e5-e028b6dd6a4e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:37.705139 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b93656d9-16a8-4b4a-93e5-e028b6dd6a4e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:37.705645 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b93656d9-16a8-4b4a-93e5-e028b6dd6a4e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:37.939269 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-284338c1-ceaa-48ea-9825-6a47587781be tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:37.940022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-284338c1-ceaa-48ea-9825-6a47587781be tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:37.972858 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-284338c1-ceaa-48ea-9825-6a47587781be tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:37.973397 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-284338c1-ceaa-48ea-9825-6a47587781be tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:37.975528 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-284338c1-ceaa-48ea-9825-6a47587781be tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:37.977835 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-284338c1-ceaa-48ea-9825-6a47587781be tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:38.063880 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b93656d9-16a8-4b4a-93e5-e028b6dd6a4e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:38.100363 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b93656d9-16a8-4b4a-93e5-e028b6dd6a4e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.409921 Jul 05 10:10:38.105186 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 41/78] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:37 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:38.115811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9d4a2427-d90c-4eb7-9738-fec9b400dbba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:38.117156 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9d4a2427-d90c-4eb7-9738-fec9b400dbba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:38.141458 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9d4a2427-d90c-4eb7-9738-fec9b400dbba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:38.143235 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9d4a2427-d90c-4eb7-9738-fec9b400dbba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:38.147420 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9d4a2427-d90c-4eb7-9738-fec9b400dbba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:38.149390 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9d4a2427-d90c-4eb7-9738-fec9b400dbba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:38.176277 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-284338c1-ceaa-48ea-9825-6a47587781be tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:38.189784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-284338c1-ceaa-48ea-9825-6a47587781be tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1299 microversion: 2.1 time: 0.257196 Jul 05 10:10:38.195203 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 38/79] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:37 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1299 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:38.278955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=16 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:38.501947 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9d4a2427-d90c-4eb7-9738-fec9b400dbba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:38.518015 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9d4a2427-d90c-4eb7-9738-fec9b400dbba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.411368 Jul 05 10:10:38.532798 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 42/80] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:38 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:38.782258 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-43972383-3375-47b0-8d89-d895cea36745 req-8ffaedc9-463b-410c-8fda-0ea8888b8487 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f6c29e5f-2d52-4c7a-9122-11dbaf998421", "tag": "494e9e9b-34e3-4ca8-b312-38f821ddd851"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:38.803292 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-43972383-3375-47b0-8d89-d895cea36745 req-8ffaedc9-463b-410c-8fda-0ea8888b8487 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:38.803749 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-43972383-3375-47b0-8d89-d895cea36745 req-8ffaedc9-463b-410c-8fda-0ea8888b8487 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:38.854990 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-43972383-3375-47b0-8d89-d895cea36745 req-8ffaedc9-463b-410c-8fda-0ea8888b8487 service nova] Creating event network-changed:494e9e9b-34e3-4ca8-b312-38f821ddd851 for instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 on hv-cinder-maste Jul 05 10:10:38.863284 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-43972383-3375-47b0-8d89-d895cea36745 req-8ffaedc9-463b-410c-8fda-0ea8888b8487 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.620917 Jul 05 10:10:38.864583 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 39/81] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:10:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:39.119177 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-7afff62a-ab5d-4a51-8391-0da8925e8c1a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:39.119790 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7afff62a-ab5d-4a51-8391-0da8925e8c1a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:39.126182 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7afff62a-ab5d-4a51-8391-0da8925e8c1a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:39.126833 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7afff62a-ab5d-4a51-8391-0da8925e8c1a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:39.127758 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7afff62a-ab5d-4a51-8391-0da8925e8c1a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:39.128417 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7afff62a-ab5d-4a51-8391-0da8925e8c1a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:39.215382 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d0147f13-5971-4ae9-a2b2-5cc1a76dccd5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:39.216638 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d0147f13-5971-4ae9-a2b2-5cc1a76dccd5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:39.223369 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d0147f13-5971-4ae9-a2b2-5cc1a76dccd5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:39.224005 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d0147f13-5971-4ae9-a2b2-5cc1a76dccd5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:39.224979 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d0147f13-5971-4ae9-a2b2-5cc1a76dccd5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:39.225581 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d0147f13-5971-4ae9-a2b2-5cc1a76dccd5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:39.392173 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-7afff62a-ab5d-4a51-8391-0da8925e8c1a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:39.400128 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-7afff62a-ab5d-4a51-8391-0da8925e8c1a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.284955 Jul 05 10:10:39.412569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 43/82] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:39 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:39.473391 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d0147f13-5971-4ae9-a2b2-5cc1a76dccd5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:39.483535 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d0147f13-5971-4ae9-a2b2-5cc1a76dccd5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.272955 Jul 05 10:10:39.485974 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 40/83] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:39 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:39.548684 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-61058aeb-ec82-4a87-b444-9d5b76bce10f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:39.549471 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-61058aeb-ec82-4a87-b444-9d5b76bce10f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:39.559220 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-61058aeb-ec82-4a87-b444-9d5b76bce10f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:39.560612 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-61058aeb-ec82-4a87-b444-9d5b76bce10f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:39.562294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-61058aeb-ec82-4a87-b444-9d5b76bce10f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:39.562735 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-61058aeb-ec82-4a87-b444-9d5b76bce10f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:39.824278 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-61058aeb-ec82-4a87-b444-9d5b76bce10f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:39.833334 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-61058aeb-ec82-4a87-b444-9d5b76bce10f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.288608 Jul 05 10:10:39.844290 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 44/84] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:39 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:40.420429 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-be47e1e2-f661-4470-bd6c-627134d020c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:40.421124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-be47e1e2-f661-4470-bd6c-627134d020c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:40.428479 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be47e1e2-f661-4470-bd6c-627134d020c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:40.429698 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be47e1e2-f661-4470-bd6c-627134d020c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:40.430775 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be47e1e2-f661-4470-bd6c-627134d020c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:40.431375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be47e1e2-f661-4470-bd6c-627134d020c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:40.503662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-adfd13e7-25f9-4039-9fc5-46da39f62f8a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:40.504912 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-adfd13e7-25f9-4039-9fc5-46da39f62f8a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:40.513411 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-adfd13e7-25f9-4039-9fc5-46da39f62f8a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:40.514256 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-adfd13e7-25f9-4039-9fc5-46da39f62f8a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:40.515230 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-adfd13e7-25f9-4039-9fc5-46da39f62f8a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:40.515815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-adfd13e7-25f9-4039-9fc5-46da39f62f8a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:40.636553 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-be47e1e2-f661-4470-bd6c-627134d020c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:40.645863 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-be47e1e2-f661-4470-bd6c-627134d020c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.229414 Jul 05 10:10:40.655954 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 41/85] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:40 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:40.691499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-adfd13e7-25f9-4039-9fc5-46da39f62f8a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:40.702758 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-adfd13e7-25f9-4039-9fc5-46da39f62f8a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.204791 Jul 05 10:10:40.704275 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 45/86] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:40 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:40.850751 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3acf8be2-7f72-4de5-8800-967de787cafe tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:40.851602 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3acf8be2-7f72-4de5-8800-967de787cafe tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:40.861143 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3acf8be2-7f72-4de5-8800-967de787cafe tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:40.874209 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3acf8be2-7f72-4de5-8800-967de787cafe tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:40.875073 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3acf8be2-7f72-4de5-8800-967de787cafe tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:40.875509 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3acf8be2-7f72-4de5-8800-967de787cafe tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:40.953592 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-ed297d9d-b1af-46f5-a4d9-67ff9125de8f req-528608d5-4ea8-48ab-ae0f-2d82e0e03812 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ab80efc4-b2fa-4761-806b-2a197cc5b00b", "tag": "66542ef5-f0db-4a7d-ab6e-ceb382db45de"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:40.965002 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-ed297d9d-b1af-46f5-a4d9-67ff9125de8f req-528608d5-4ea8-48ab-ae0f-2d82e0e03812 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:40.965673 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-ed297d9d-b1af-46f5-a4d9-67ff9125de8f req-528608d5-4ea8-48ab-ae0f-2d82e0e03812 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:40.981071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-ed297d9d-b1af-46f5-a4d9-67ff9125de8f req-528608d5-4ea8-48ab-ae0f-2d82e0e03812 service nova] Creating event network-changed:66542ef5-f0db-4a7d-ab6e-ceb382db45de for instance ab80efc4-b2fa-4761-806b-2a197cc5b00b on hv-cinder-maste Jul 05 10:10:40.986543 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-ed297d9d-b1af-46f5-a4d9-67ff9125de8f req-528608d5-4ea8-48ab-ae0f-2d82e0e03812 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037305 Jul 05 10:10:40.988065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 46/87] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:10:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:41.076469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-3acf8be2-7f72-4de5-8800-967de787cafe tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:41.084282 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3acf8be2-7f72-4de5-8800-967de787cafe tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.237236 Jul 05 10:10:41.084921 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 42/88] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:40 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:41.666148 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-14ad7ac7-8d79-4bf4-a6be-b2bf0cc61c34 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:41.666984 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-14ad7ac7-8d79-4bf4-a6be-b2bf0cc61c34 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:41.681165 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-14ad7ac7-8d79-4bf4-a6be-b2bf0cc61c34 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:41.684536 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-14ad7ac7-8d79-4bf4-a6be-b2bf0cc61c34 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:41.685837 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-14ad7ac7-8d79-4bf4-a6be-b2bf0cc61c34 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:41.686283 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-14ad7ac7-8d79-4bf4-a6be-b2bf0cc61c34 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:41.721773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6f52f60e-f864-493f-9c0f-55d405ddbdb0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:41.723010 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6f52f60e-f864-493f-9c0f-55d405ddbdb0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:41.736305 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6f52f60e-f864-493f-9c0f-55d405ddbdb0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:41.738052 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6f52f60e-f864-493f-9c0f-55d405ddbdb0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:41.745300 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6f52f60e-f864-493f-9c0f-55d405ddbdb0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:41.746314 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6f52f60e-f864-493f-9c0f-55d405ddbdb0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:42.102633 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6f52f60e-f864-493f-9c0f-55d405ddbdb0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:42.112138 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6f52f60e-f864-493f-9c0f-55d405ddbdb0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.395421 Jul 05 10:10:42.113673 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 43/89] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:41 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:42.125190 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e364c6da-d881-4ca1-88f0-cd0eb3e30d08 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:42.128875 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e364c6da-d881-4ca1-88f0-cd0eb3e30d08 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:42.147030 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e364c6da-d881-4ca1-88f0-cd0eb3e30d08 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:42.153091 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e364c6da-d881-4ca1-88f0-cd0eb3e30d08 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:42.154420 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e364c6da-d881-4ca1-88f0-cd0eb3e30d08 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:42.154740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e364c6da-d881-4ca1-88f0-cd0eb3e30d08 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:42.185224 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-14ad7ac7-8d79-4bf4-a6be-b2bf0cc61c34 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:42.189648 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=100 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:42.198410 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-14ad7ac7-8d79-4bf4-a6be-b2bf0cc61c34 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.538869 Jul 05 10:10:42.199686 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 47/90] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:41 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:42.322994 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e364c6da-d881-4ca1-88f0-cd0eb3e30d08 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:42.332021 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e364c6da-d881-4ca1-88f0-cd0eb3e30d08 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.215350 Jul 05 10:10:42.332903 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 44/91] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:42 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:43.129351 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6391f193-c917-44d4-a920-8505c58de82d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:43.129959 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-6391f193-c917-44d4-a920-8505c58de82d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:43.132057 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=6 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:43.137261 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6391f193-c917-44d4-a920-8505c58de82d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:43.137885 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6391f193-c917-44d4-a920-8505c58de82d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:43.138655 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6391f193-c917-44d4-a920-8505c58de82d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:43.139093 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6391f193-c917-44d4-a920-8505c58de82d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:43.220326 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8bcbe329-6556-4509-895a-10f46190f63f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:43.220326 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-8bcbe329-6556-4509-895a-10f46190f63f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:43.220326 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=22 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:43.229704 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8bcbe329-6556-4509-895a-10f46190f63f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:43.230705 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8bcbe329-6556-4509-895a-10f46190f63f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:43.231795 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8bcbe329-6556-4509-895a-10f46190f63f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:43.232301 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8bcbe329-6556-4509-895a-10f46190f63f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:43.354167 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6391f193-c917-44d4-a920-8505c58de82d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:43.363743 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-6391f193-c917-44d4-a920-8505c58de82d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.238096 Jul 05 10:10:43.366485 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 48/92] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:43 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:43.374025 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-154e946b-d535-4ba3-a018-fb7b7c4fff9c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:43.374840 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-154e946b-d535-4ba3-a018-fb7b7c4fff9c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:43.390946 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-154e946b-d535-4ba3-a018-fb7b7c4fff9c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:43.392950 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-154e946b-d535-4ba3-a018-fb7b7c4fff9c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:43.395821 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-154e946b-d535-4ba3-a018-fb7b7c4fff9c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:43.397038 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-154e946b-d535-4ba3-a018-fb7b7c4fff9c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:43.420535 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8bcbe329-6556-4509-895a-10f46190f63f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:43.424543 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=100 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:43.432714 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-8bcbe329-6556-4509-895a-10f46190f63f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.220158 Jul 05 10:10:43.434204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 45/93] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:43 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:43.594924 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-154e946b-d535-4ba3-a018-fb7b7c4fff9c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:43.614075 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-154e946b-d535-4ba3-a018-fb7b7c4fff9c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.246521 Jul 05 10:10:43.614838 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 49/94] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:43 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:44.383112 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f124343a-d798-4324-abc7-6c396044793b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:44.384445 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f124343a-d798-4324-abc7-6c396044793b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:44.386374 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=86,nova_api:INSERT=7 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:44.393439 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f124343a-d798-4324-abc7-6c396044793b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:44.394141 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f124343a-d798-4324-abc7-6c396044793b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:44.394932 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f124343a-d798-4324-abc7-6c396044793b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:44.395370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f124343a-d798-4324-abc7-6c396044793b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:44.450588 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-65a8b534-4c8f-4150-a1e8-9372a9e88cb3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:44.451966 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-65a8b534-4c8f-4150-a1e8-9372a9e88cb3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:44.454212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=6 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:44.459890 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-65a8b534-4c8f-4150-a1e8-9372a9e88cb3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:44.460727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-65a8b534-4c8f-4150-a1e8-9372a9e88cb3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:44.461707 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-65a8b534-4c8f-4150-a1e8-9372a9e88cb3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:44.462339 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-65a8b534-4c8f-4150-a1e8-9372a9e88cb3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:44.629083 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f124343a-d798-4324-abc7-6c396044793b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:44.644984 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f124343a-d798-4324-abc7-6c396044793b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.267134 Jul 05 10:10:44.645637 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 46/95] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:44 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:44.650502 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-1ea90542-ac8d-41e2-bd41-c5905576f34b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:44.651173 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-1ea90542-ac8d-41e2-bd41-c5905576f34b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:44.658321 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1ea90542-ac8d-41e2-bd41-c5905576f34b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:44.659015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1ea90542-ac8d-41e2-bd41-c5905576f34b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:44.660080 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1ea90542-ac8d-41e2-bd41-c5905576f34b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:44.660941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1ea90542-ac8d-41e2-bd41-c5905576f34b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:44.663373 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-65a8b534-4c8f-4150-a1e8-9372a9e88cb3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:44.671428 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-65a8b534-4c8f-4150-a1e8-9372a9e88cb3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.225113 Jul 05 10:10:44.672105 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 50/96] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:44 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:44.804769 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1ea90542-ac8d-41e2-bd41-c5905576f34b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:44.812413 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-1ea90542-ac8d-41e2-bd41-c5905576f34b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.165630 Jul 05 10:10:44.812980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 47/97] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:44 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:45.662682 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c57421aa-47dc-47af-b90d-bcff2259ad89 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:45.663888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c57421aa-47dc-47af-b90d-bcff2259ad89 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:45.669783 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c57421aa-47dc-47af-b90d-bcff2259ad89 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:45.670229 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c57421aa-47dc-47af-b90d-bcff2259ad89 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:45.670986 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c57421aa-47dc-47af-b90d-bcff2259ad89 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:45.671424 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c57421aa-47dc-47af-b90d-bcff2259ad89 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:45.689102 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a36364c4-3554-4092-b97f-4fe1473b6952 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:45.689908 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a36364c4-3554-4092-b97f-4fe1473b6952 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:45.697347 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a36364c4-3554-4092-b97f-4fe1473b6952 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:45.698137 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a36364c4-3554-4092-b97f-4fe1473b6952 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:45.699165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a36364c4-3554-4092-b97f-4fe1473b6952 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:45.699828 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a36364c4-3554-4092-b97f-4fe1473b6952 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:45.989366 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a36364c4-3554-4092-b97f-4fe1473b6952 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:45.996871 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a36364c4-3554-4092-b97f-4fe1473b6952 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.311697 Jul 05 10:10:45.997356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 48/98] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:45 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:46.003933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-30e6fc94-999f-450e-9f33-8152750f6ea0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:46.004758 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-30e6fc94-999f-450e-9f33-8152750f6ea0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:46.015617 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-30e6fc94-999f-450e-9f33-8152750f6ea0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:46.016077 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-30e6fc94-999f-450e-9f33-8152750f6ea0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:46.016383 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c57421aa-47dc-47af-b90d-bcff2259ad89 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:46.023413 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-30e6fc94-999f-450e-9f33-8152750f6ea0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:46.024180 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-30e6fc94-999f-450e-9f33-8152750f6ea0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:46.027831 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c57421aa-47dc-47af-b90d-bcff2259ad89 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.369440 Jul 05 10:10:46.041994 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 51/99] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:45 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:46.212353 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-30e6fc94-999f-450e-9f33-8152750f6ea0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:46.224643 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-30e6fc94-999f-450e-9f33-8152750f6ea0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.224100 Jul 05 10:10:46.234262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 49/100] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:45 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:47.013726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-24153551-87eb-4e3e-aaf7-bdc8d0e342d1 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:47.014268 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-24153551-87eb-4e3e-aaf7-bdc8d0e342d1 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:47.022087 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-24153551-87eb-4e3e-aaf7-bdc8d0e342d1 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:47.022811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-24153551-87eb-4e3e-aaf7-bdc8d0e342d1 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:47.023529 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-24153551-87eb-4e3e-aaf7-bdc8d0e342d1 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:47.024329 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-24153551-87eb-4e3e-aaf7-bdc8d0e342d1 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:47.047037 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c0828ba1-83eb-4bdd-b090-ccafaba18a31 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:47.047746 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-c0828ba1-83eb-4bdd-b090-ccafaba18a31 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:47.056537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c0828ba1-83eb-4bdd-b090-ccafaba18a31 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:47.057162 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c0828ba1-83eb-4bdd-b090-ccafaba18a31 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:47.062793 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c0828ba1-83eb-4bdd-b090-ccafaba18a31 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:47.067190 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c0828ba1-83eb-4bdd-b090-ccafaba18a31 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:47.268626 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-c0828ba1-83eb-4bdd-b090-ccafaba18a31 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:47.282621 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-c0828ba1-83eb-4bdd-b090-ccafaba18a31 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.237852 Jul 05 10:10:47.283648 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 50/101] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:47 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:47.284839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-24153551-87eb-4e3e-aaf7-bdc8d0e342d1 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:47.289182 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e946c4ce-901b-4c0a-96be-bf584a311306 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:47.289800 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e946c4ce-901b-4c0a-96be-bf584a311306 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:47.292480 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-24153551-87eb-4e3e-aaf7-bdc8d0e342d1 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.282470 Jul 05 10:10:47.295930 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e946c4ce-901b-4c0a-96be-bf584a311306 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:47.296500 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e946c4ce-901b-4c0a-96be-bf584a311306 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:47.297500 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e946c4ce-901b-4c0a-96be-bf584a311306 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:47.298435 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e946c4ce-901b-4c0a-96be-bf584a311306 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:47.305851 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 52/102] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:47 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:47.394472 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=6 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:47.486622 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e946c4ce-901b-4c0a-96be-bf584a311306 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:47.496527 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e946c4ce-901b-4c0a-96be-bf584a311306 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.211554 Jul 05 10:10:47.497204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 51/103] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:47 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:48.300099 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d8ea2c53-35fd-43ac-a2c9-8d51296db8b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:48.300664 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d8ea2c53-35fd-43ac-a2c9-8d51296db8b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:48.308103 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8ea2c53-35fd-43ac-a2c9-8d51296db8b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:48.308103 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8ea2c53-35fd-43ac-a2c9-8d51296db8b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:48.308103 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8ea2c53-35fd-43ac-a2c9-8d51296db8b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:48.308302 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8ea2c53-35fd-43ac-a2c9-8d51296db8b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:48.321161 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7f770360-7ce9-4e0a-a571-da43d5bf3964 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:48.321548 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7f770360-7ce9-4e0a-a571-da43d5bf3964 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:48.329833 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7f770360-7ce9-4e0a-a571-da43d5bf3964 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:48.330294 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7f770360-7ce9-4e0a-a571-da43d5bf3964 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:48.332235 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7f770360-7ce9-4e0a-a571-da43d5bf3964 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:48.332711 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7f770360-7ce9-4e0a-a571-da43d5bf3964 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:48.668626 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d8ea2c53-35fd-43ac-a2c9-8d51296db8b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:48.671395 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-7f770360-7ce9-4e0a-a571-da43d5bf3964 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:48.675937 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d8ea2c53-35fd-43ac-a2c9-8d51296db8b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.379640 Jul 05 10:10:48.676731 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 53/104] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:48 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:48.678871 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7f770360-7ce9-4e0a-a571-da43d5bf3964 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.369091 Jul 05 10:10:48.679421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 52/105] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:48 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:48.682210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-4f9bd939-883e-48ba-af31-17470b647474 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:48.682823 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-4f9bd939-883e-48ba-af31-17470b647474 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:48.688716 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f9bd939-883e-48ba-af31-17470b647474 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:48.689162 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f9bd939-883e-48ba-af31-17470b647474 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:48.689909 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f9bd939-883e-48ba-af31-17470b647474 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:48.690339 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f9bd939-883e-48ba-af31-17470b647474 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:48.813944 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4f9bd939-883e-48ba-af31-17470b647474 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:48.821854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-4f9bd939-883e-48ba-af31-17470b647474 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.143438 Jul 05 10:10:48.822678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 54/106] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:48 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:49.692593 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fd29844b-2311-435e-9751-bbfbf6a7155d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:49.693207 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fd29844b-2311-435e-9751-bbfbf6a7155d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:49.695546 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8fe149c9-4df6-437b-a75e-459c3300467e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:49.696152 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8fe149c9-4df6-437b-a75e-459c3300467e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:49.700029 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fd29844b-2311-435e-9751-bbfbf6a7155d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:49.700517 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fd29844b-2311-435e-9751-bbfbf6a7155d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:49.701286 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fd29844b-2311-435e-9751-bbfbf6a7155d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:49.701723 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fd29844b-2311-435e-9751-bbfbf6a7155d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:49.702373 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8fe149c9-4df6-437b-a75e-459c3300467e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:49.702813 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8fe149c9-4df6-437b-a75e-459c3300467e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:49.703565 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8fe149c9-4df6-437b-a75e-459c3300467e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:49.703998 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8fe149c9-4df6-437b-a75e-459c3300467e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:49.889485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fd29844b-2311-435e-9751-bbfbf6a7155d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:49.896648 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8fe149c9-4df6-437b-a75e-459c3300467e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:49.896970 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fd29844b-2311-435e-9751-bbfbf6a7155d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.208383 Jul 05 10:10:49.897593 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 53/107] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:49 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:49.902034 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-96b0a1cb-1646-4c9d-aea1-76fbcb2143d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:49.902644 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-96b0a1cb-1646-4c9d-aea1-76fbcb2143d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:49.905392 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8fe149c9-4df6-437b-a75e-459c3300467e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.213829 Jul 05 10:10:49.905889 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 55/108] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:49 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:49.908415 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-96b0a1cb-1646-4c9d-aea1-76fbcb2143d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:49.908876 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-96b0a1cb-1646-4c9d-aea1-76fbcb2143d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:49.909643 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-96b0a1cb-1646-4c9d-aea1-76fbcb2143d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:49.910086 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-96b0a1cb-1646-4c9d-aea1-76fbcb2143d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:50.040201 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-96b0a1cb-1646-4c9d-aea1-76fbcb2143d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:50.047568 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-96b0a1cb-1646-4c9d-aea1-76fbcb2143d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.149040 Jul 05 10:10:50.048126 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 54/109] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:49 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:50.913573 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-568f773c-8069-42a8-8513-656de252c807 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:50.914227 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-568f773c-8069-42a8-8513-656de252c807 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:50.920664 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-568f773c-8069-42a8-8513-656de252c807 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:50.921114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-568f773c-8069-42a8-8513-656de252c807 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:50.921645 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-193cb4e0-112e-48e2-b561-266779de6018 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:50.922030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-568f773c-8069-42a8-8513-656de252c807 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:50.922259 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-193cb4e0-112e-48e2-b561-266779de6018 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:50.922365 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-568f773c-8069-42a8-8513-656de252c807 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:50.928477 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-193cb4e0-112e-48e2-b561-266779de6018 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:50.928827 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-193cb4e0-112e-48e2-b561-266779de6018 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:50.929598 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-193cb4e0-112e-48e2-b561-266779de6018 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:50.930089 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-193cb4e0-112e-48e2-b561-266779de6018 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:51.115062 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-568f773c-8069-42a8-8513-656de252c807 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:51.119143 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-193cb4e0-112e-48e2-b561-266779de6018 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:51.122620 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-568f773c-8069-42a8-8513-656de252c807 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.212898 Jul 05 10:10:51.123198 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 56/110] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:50 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:51.126741 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-193cb4e0-112e-48e2-b561-266779de6018 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.208858 Jul 05 10:10:51.127286 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 55/111] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:50 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:51.133214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9f420a0b-1d14-461a-911f-1ca20efeaeed tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:51.133842 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9f420a0b-1d14-461a-911f-1ca20efeaeed tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:51.139595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9f420a0b-1d14-461a-911f-1ca20efeaeed tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:51.140044 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9f420a0b-1d14-461a-911f-1ca20efeaeed tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:51.140841 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9f420a0b-1d14-461a-911f-1ca20efeaeed tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:51.141278 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9f420a0b-1d14-461a-911f-1ca20efeaeed tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:51.270316 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9f420a0b-1d14-461a-911f-1ca20efeaeed tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:51.277600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9f420a0b-1d14-461a-911f-1ca20efeaeed tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.148607 Jul 05 10:10:51.278171 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 57/112] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:51 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:52.138996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-4d84ddd1-138a-4063-a16f-8939e23b417a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:52.139600 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-4d84ddd1-138a-4063-a16f-8939e23b417a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:52.142714 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-130e5eff-fcc0-4e08-a3fc-21658d8137ed tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:52.143452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-130e5eff-fcc0-4e08-a3fc-21658d8137ed tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:52.145564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4d84ddd1-138a-4063-a16f-8939e23b417a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:52.146006 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4d84ddd1-138a-4063-a16f-8939e23b417a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:52.146758 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4d84ddd1-138a-4063-a16f-8939e23b417a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:52.147184 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4d84ddd1-138a-4063-a16f-8939e23b417a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:52.149869 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-130e5eff-fcc0-4e08-a3fc-21658d8137ed tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:52.150317 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-130e5eff-fcc0-4e08-a3fc-21658d8137ed tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:52.151072 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-130e5eff-fcc0-4e08-a3fc-21658d8137ed tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:52.151511 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-130e5eff-fcc0-4e08-a3fc-21658d8137ed tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:52.337285 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-4d84ddd1-138a-4063-a16f-8939e23b417a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:52.341342 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-130e5eff-fcc0-4e08-a3fc-21658d8137ed tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:52.344572 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-4d84ddd1-138a-4063-a16f-8939e23b417a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.209494 Jul 05 10:10:52.345139 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 56/113] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:52 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:52.348800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-130e5eff-fcc0-4e08-a3fc-21658d8137ed tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.209774 Jul 05 10:10:52.352432 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 58/114] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:52 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:52.352554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-57cd92ea-965c-4bd5-869a-d63765c30e8f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:52.352554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-57cd92ea-965c-4bd5-869a-d63765c30e8f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:52.356360 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-57cd92ea-965c-4bd5-869a-d63765c30e8f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:52.356830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-57cd92ea-965c-4bd5-869a-d63765c30e8f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:52.357582 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-57cd92ea-965c-4bd5-869a-d63765c30e8f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:52.358012 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-57cd92ea-965c-4bd5-869a-d63765c30e8f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:52.483704 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-57cd92ea-965c-4bd5-869a-d63765c30e8f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:52.491068 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-57cd92ea-965c-4bd5-869a-d63765c30e8f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.145002 Jul 05 10:10:52.491780 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 57/115] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:52 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:53.362028 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a15c92ef-b1f5-411a-9c37-ef9c094d7274 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:53.362641 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a15c92ef-b1f5-411a-9c37-ef9c094d7274 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:53.365747 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3ffae460-1271-4734-8cf7-df9e3297601a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:53.366359 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3ffae460-1271-4734-8cf7-df9e3297601a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:53.369340 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a15c92ef-b1f5-411a-9c37-ef9c094d7274 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:53.369793 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a15c92ef-b1f5-411a-9c37-ef9c094d7274 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:53.370562 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a15c92ef-b1f5-411a-9c37-ef9c094d7274 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:53.370997 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a15c92ef-b1f5-411a-9c37-ef9c094d7274 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:53.373028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3ffae460-1271-4734-8cf7-df9e3297601a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:53.373484 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3ffae460-1271-4734-8cf7-df9e3297601a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:53.374243 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3ffae460-1271-4734-8cf7-df9e3297601a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:53.374683 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3ffae460-1271-4734-8cf7-df9e3297601a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:53.561530 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a15c92ef-b1f5-411a-9c37-ef9c094d7274 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:53.565695 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-3ffae460-1271-4734-8cf7-df9e3297601a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:53.568993 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a15c92ef-b1f5-411a-9c37-ef9c094d7274 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.211021 Jul 05 10:10:53.569571 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 59/116] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:53 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:53.573697 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3ffae460-1271-4734-8cf7-df9e3297601a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.211663 Jul 05 10:10:53.574388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-adf1fef7-bc28-4b02-bc3c-cbb42af2a0d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:53.574506 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 58/117] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:53 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:53.574874 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-adf1fef7-bc28-4b02-bc3c-cbb42af2a0d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:53.580872 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-adf1fef7-bc28-4b02-bc3c-cbb42af2a0d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:53.581333 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-adf1fef7-bc28-4b02-bc3c-cbb42af2a0d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:53.582123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-adf1fef7-bc28-4b02-bc3c-cbb42af2a0d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:53.582554 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-adf1fef7-bc28-4b02-bc3c-cbb42af2a0d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:53.713633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-adf1fef7-bc28-4b02-bc3c-cbb42af2a0d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:53.722384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-adf1fef7-bc28-4b02-bc3c-cbb42af2a0d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.151686 Jul 05 10:10:53.722985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 60/118] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:53 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:54.585867 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-33c0d472-5738-497d-92f7-5542ecddfc6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:54.586941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-33c0d472-5738-497d-92f7-5542ecddfc6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:54.593180 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-33c0d472-5738-497d-92f7-5542ecddfc6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:54.593628 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-33c0d472-5738-497d-92f7-5542ecddfc6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:54.593739 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c571f930-90cd-4169-9e07-dc35a4d85531 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:54.594294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c571f930-90cd-4169-9e07-dc35a4d85531 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:54.594390 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-33c0d472-5738-497d-92f7-5542ecddfc6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:54.594845 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-33c0d472-5738-497d-92f7-5542ecddfc6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:54.600307 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c571f930-90cd-4169-9e07-dc35a4d85531 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:54.600777 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c571f930-90cd-4169-9e07-dc35a4d85531 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:54.601534 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c571f930-90cd-4169-9e07-dc35a4d85531 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:54.602658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c571f930-90cd-4169-9e07-dc35a4d85531 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:54.793847 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-33c0d472-5738-497d-92f7-5542ecddfc6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:54.797434 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c571f930-90cd-4169-9e07-dc35a4d85531 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:54.801389 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-33c0d472-5738-497d-92f7-5542ecddfc6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.219366 Jul 05 10:10:54.801960 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 59/119] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:54 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:54.805107 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c571f930-90cd-4169-9e07-dc35a4d85531 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.215179 Jul 05 10:10:54.805794 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 61/120] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:54 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:54.806332 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-684566dd-348d-407d-96a5-1777e411c743 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:54.806946 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-684566dd-348d-407d-96a5-1777e411c743 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:54.812670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-684566dd-348d-407d-96a5-1777e411c743 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:54.813128 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-684566dd-348d-407d-96a5-1777e411c743 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:54.813893 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-684566dd-348d-407d-96a5-1777e411c743 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:54.814337 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-684566dd-348d-407d-96a5-1777e411c743 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:54.942802 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-684566dd-348d-407d-96a5-1777e411c743 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:54.951066 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-684566dd-348d-407d-96a5-1777e411c743 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.148170 Jul 05 10:10:54.951625 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 60/121] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:54 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:55.817917 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a4e6d397-bb1b-433f-88c6-1c8eabd5f871 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:55.818529 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a4e6d397-bb1b-433f-88c6-1c8eabd5f871 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:55.821398 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-dd3ab13c-aa7b-4881-b49a-aa90757e7754 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:55.822002 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-dd3ab13c-aa7b-4881-b49a-aa90757e7754 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:55.824850 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a4e6d397-bb1b-433f-88c6-1c8eabd5f871 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:55.825304 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a4e6d397-bb1b-433f-88c6-1c8eabd5f871 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:55.826059 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a4e6d397-bb1b-433f-88c6-1c8eabd5f871 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:55.826492 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a4e6d397-bb1b-433f-88c6-1c8eabd5f871 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:55.828147 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dd3ab13c-aa7b-4881-b49a-aa90757e7754 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:55.828627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dd3ab13c-aa7b-4881-b49a-aa90757e7754 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:55.829381 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dd3ab13c-aa7b-4881-b49a-aa90757e7754 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:55.829815 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dd3ab13c-aa7b-4881-b49a-aa90757e7754 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:56.018328 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a4e6d397-bb1b-433f-88c6-1c8eabd5f871 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:56.021674 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-dd3ab13c-aa7b-4881-b49a-aa90757e7754 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:56.026279 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a4e6d397-bb1b-433f-88c6-1c8eabd5f871 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.212241 Jul 05 10:10:56.026935 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 62/122] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:55 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:56.029544 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-dd3ab13c-aa7b-4881-b49a-aa90757e7754 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.211847 Jul 05 10:10:56.030095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 61/123] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:55 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:56.031648 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0ba14f26-6514-44e5-9d7e-b12f78428ea1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:56.032320 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0ba14f26-6514-44e5-9d7e-b12f78428ea1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:56.038352 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0ba14f26-6514-44e5-9d7e-b12f78428ea1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:56.038798 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0ba14f26-6514-44e5-9d7e-b12f78428ea1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:56.040198 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0ba14f26-6514-44e5-9d7e-b12f78428ea1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:56.040815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0ba14f26-6514-44e5-9d7e-b12f78428ea1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:56.169996 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0ba14f26-6514-44e5-9d7e-b12f78428ea1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:56.178649 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0ba14f26-6514-44e5-9d7e-b12f78428ea1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.150681 Jul 05 10:10:56.179460 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 63/124] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:56 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:56.452587 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-43954445-47ec-4b79-ba02-294251d16b34 req-9ced89a7-5b39-4c33-8b12-a267fb0788da service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b7ce1e24-0c58-4071-9610-57d072131857", "name": "network-vif-plugged", "status": "completed", "tag": "1590f070-4b3b-4762-9ba6-291ca1720666"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:10:56.460565 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-43954445-47ec-4b79-ba02-294251d16b34 req-9ced89a7-5b39-4c33-8b12-a267fb0788da service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:56.461009 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-43954445-47ec-4b79-ba02-294251d16b34 req-9ced89a7-5b39-4c33-8b12-a267fb0788da service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:56.472550 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-43954445-47ec-4b79-ba02-294251d16b34 req-9ced89a7-5b39-4c33-8b12-a267fb0788da service nova] Creating event network-vif-plugged:1590f070-4b3b-4762-9ba6-291ca1720666 for instance b7ce1e24-0c58-4071-9610-57d072131857 on hv-cinder-maste Jul 05 10:10:56.477884 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-43954445-47ec-4b79-ba02-294251d16b34 req-9ced89a7-5b39-4c33-8b12-a267fb0788da service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083926 Jul 05 10:10:56.478451 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 62/125] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:10:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:10:57.043042 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c4031b24-d40b-4103-bc94-9bd028e97526 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:57.043722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c4031b24-d40b-4103-bc94-9bd028e97526 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:57.046229 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e07a4f36-f602-42a8-a1ff-58c66e3f1514 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:57.046926 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e07a4f36-f602-42a8-a1ff-58c66e3f1514 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:57.049803 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c4031b24-d40b-4103-bc94-9bd028e97526 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:57.050258 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c4031b24-d40b-4103-bc94-9bd028e97526 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:57.051020 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c4031b24-d40b-4103-bc94-9bd028e97526 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:57.051463 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c4031b24-d40b-4103-bc94-9bd028e97526 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:57.053078 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e07a4f36-f602-42a8-a1ff-58c66e3f1514 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:57.053525 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e07a4f36-f602-42a8-a1ff-58c66e3f1514 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:57.054283 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e07a4f36-f602-42a8-a1ff-58c66e3f1514 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:57.054717 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e07a4f36-f602-42a8-a1ff-58c66e3f1514 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:57.058784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=100 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:57.069207 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=100 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:10:57.245217 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c4031b24-d40b-4103-bc94-9bd028e97526 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:57.250664 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e07a4f36-f602-42a8-a1ff-58c66e3f1514 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:57.252978 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c4031b24-d40b-4103-bc94-9bd028e97526 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.213879 Jul 05 10:10:57.253556 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 64/126] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:57 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:57.258139 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-133b3816-7f4d-4e2b-ab65-4e13259307d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:57.259301 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-133b3816-7f4d-4e2b-ab65-4e13259307d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:57.265387 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-133b3816-7f4d-4e2b-ab65-4e13259307d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:57.265835 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-133b3816-7f4d-4e2b-ab65-4e13259307d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:57.266782 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-133b3816-7f4d-4e2b-ab65-4e13259307d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:57.267222 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-133b3816-7f4d-4e2b-ab65-4e13259307d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:57.268746 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e07a4f36-f602-42a8-a1ff-58c66e3f1514 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.226544 Jul 05 10:10:57.269472 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 63/127] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:57 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:57.394402 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-133b3816-7f4d-4e2b-ab65-4e13259307d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:57.401688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-133b3816-7f4d-4e2b-ab65-4e13259307d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.146995 Jul 05 10:10:57.402256 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 65/128] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:57 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:58.270358 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-32a26d17-dc4f-4ddd-a3a3-9d23b7df55e0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:58.270889 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-32a26d17-dc4f-4ddd-a3a3-9d23b7df55e0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:58.277431 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-32a26d17-dc4f-4ddd-a3a3-9d23b7df55e0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:58.277905 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-32a26d17-dc4f-4ddd-a3a3-9d23b7df55e0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:58.278655 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-32a26d17-dc4f-4ddd-a3a3-9d23b7df55e0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:58.279086 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-32a26d17-dc4f-4ddd-a3a3-9d23b7df55e0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:58.285104 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d873e384-da4e-4c05-becd-446e68ccc86d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:58.285711 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d873e384-da4e-4c05-becd-446e68ccc86d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:58.292213 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d873e384-da4e-4c05-becd-446e68ccc86d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:58.292696 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d873e384-da4e-4c05-becd-446e68ccc86d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:58.293462 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d873e384-da4e-4c05-becd-446e68ccc86d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:58.293898 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d873e384-da4e-4c05-becd-446e68ccc86d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:58.480865 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-32a26d17-dc4f-4ddd-a3a3-9d23b7df55e0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:58.483998 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d873e384-da4e-4c05-becd-446e68ccc86d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:58.488580 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-32a26d17-dc4f-4ddd-a3a3-9d23b7df55e0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.222052 Jul 05 10:10:58.489339 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 64/129] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:58 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:58.491399 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d873e384-da4e-4c05-becd-446e68ccc86d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.210062 Jul 05 10:10:58.491953 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 66/130] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:58 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:58.497894 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d1495003-ca5a-4a30-bebf-0173442751ca tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:58.498524 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d1495003-ca5a-4a30-bebf-0173442751ca tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:58.504668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d1495003-ca5a-4a30-bebf-0173442751ca tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:58.505122 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d1495003-ca5a-4a30-bebf-0173442751ca tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:58.505902 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d1495003-ca5a-4a30-bebf-0173442751ca tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:58.506338 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d1495003-ca5a-4a30-bebf-0173442751ca tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:58.642619 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d1495003-ca5a-4a30-bebf-0173442751ca tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:58.650833 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d1495003-ca5a-4a30-bebf-0173442751ca tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.157695 Jul 05 10:10:58.651463 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 65/131] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:58 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:59.505177 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a38ba441-d35c-46ff-bcbd-ca8567493b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:59.505975 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a38ba441-d35c-46ff-bcbd-ca8567493b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:59.506954 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5ed5f9ff-29a8-406e-8172-f3ec7f05dd12 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:59.507715 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5ed5f9ff-29a8-406e-8172-f3ec7f05dd12 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:59.512549 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a38ba441-d35c-46ff-bcbd-ca8567493b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:59.513004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a38ba441-d35c-46ff-bcbd-ca8567493b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:59.513651 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5ed5f9ff-29a8-406e-8172-f3ec7f05dd12 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:59.513799 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a38ba441-d35c-46ff-bcbd-ca8567493b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:59.514202 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a38ba441-d35c-46ff-bcbd-ca8567493b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:59.514374 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5ed5f9ff-29a8-406e-8172-f3ec7f05dd12 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:59.515127 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5ed5f9ff-29a8-406e-8172-f3ec7f05dd12 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:59.515714 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5ed5f9ff-29a8-406e-8172-f3ec7f05dd12 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:59.710663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a38ba441-d35c-46ff-bcbd-ca8567493b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:59.714323 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5ed5f9ff-29a8-406e-8172-f3ec7f05dd12 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:59.719251 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a38ba441-d35c-46ff-bcbd-ca8567493b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.217961 Jul 05 10:10:59.720039 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 67/132] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:59 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:10:59.722896 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5ed5f9ff-29a8-406e-8172-f3ec7f05dd12 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.219775 Jul 05 10:10:59.723696 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 66/133] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:59 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:10:59.724288 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b40eae51-ddf1-4631-b90b-1be09c2f8202 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:10:59.724898 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b40eae51-ddf1-4631-b90b-1be09c2f8202 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:10:59.731788 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b40eae51-ddf1-4631-b90b-1be09c2f8202 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:59.732227 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b40eae51-ddf1-4631-b90b-1be09c2f8202 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:59.733016 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b40eae51-ddf1-4631-b90b-1be09c2f8202 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:10:59.733446 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b40eae51-ddf1-4631-b90b-1be09c2f8202 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:10:59.863344 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b40eae51-ddf1-4631-b90b-1be09c2f8202 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:10:59.870688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b40eae51-ddf1-4631-b90b-1be09c2f8202 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.149936 Jul 05 10:10:59.871241 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 68/134] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:10:59 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:00.736985 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-919b7a37-87de-4b90-a59b-b103e1a203b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:00.737601 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-919b7a37-87de-4b90-a59b-b103e1a203b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:00.739709 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-06bd4ad8-86d7-4d82-bc70-327bf416c4dd tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:00.740814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-06bd4ad8-86d7-4d82-bc70-327bf416c4dd tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:00.744508 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-919b7a37-87de-4b90-a59b-b103e1a203b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:00.744953 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-919b7a37-87de-4b90-a59b-b103e1a203b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:00.745708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-919b7a37-87de-4b90-a59b-b103e1a203b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:00.746148 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-919b7a37-87de-4b90-a59b-b103e1a203b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:00.747342 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-06bd4ad8-86d7-4d82-bc70-327bf416c4dd tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:00.747783 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-06bd4ad8-86d7-4d82-bc70-327bf416c4dd tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:00.748579 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-06bd4ad8-86d7-4d82-bc70-327bf416c4dd tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:00.749009 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-06bd4ad8-86d7-4d82-bc70-327bf416c4dd tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:00.983442 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-919b7a37-87de-4b90-a59b-b103e1a203b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:00.986159 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-06bd4ad8-86d7-4d82-bc70-327bf416c4dd tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:00.991477 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-919b7a37-87de-4b90-a59b-b103e1a203b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.258809 Jul 05 10:11:00.992403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 67/135] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:00 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:00.994178 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-06bd4ad8-86d7-4d82-bc70-327bf416c4dd tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.258311 Jul 05 10:11:00.994739 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 69/136] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:00 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:00.997535 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-9d578b4c-3b37-466c-89d7-5a3ae111eee0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:00.998134 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-9d578b4c-3b37-466c-89d7-5a3ae111eee0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:01.003721 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9d578b4c-3b37-466c-89d7-5a3ae111eee0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:01.004165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9d578b4c-3b37-466c-89d7-5a3ae111eee0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:01.004953 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9d578b4c-3b37-466c-89d7-5a3ae111eee0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:01.005385 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9d578b4c-3b37-466c-89d7-5a3ae111eee0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:01.133596 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-9d578b4c-3b37-466c-89d7-5a3ae111eee0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:01.140880 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-9d578b4c-3b37-466c-89d7-5a3ae111eee0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.146768 Jul 05 10:11:01.141433 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 68/137] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:00 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:02.010536 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a2574190-f756-4f72-9aeb-e209b52b5af0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:02.011684 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-79ae6975-4bb8-48b8-98ca-115a93e3c44e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:02.012222 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a2574190-f756-4f72-9aeb-e209b52b5af0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:02.012464 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-79ae6975-4bb8-48b8-98ca-115a93e3c44e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:02.018746 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-79ae6975-4bb8-48b8-98ca-115a93e3c44e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:02.019383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-79ae6975-4bb8-48b8-98ca-115a93e3c44e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:02.020316 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-79ae6975-4bb8-48b8-98ca-115a93e3c44e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:02.020848 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a2574190-f756-4f72-9aeb-e209b52b5af0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:02.021118 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-79ae6975-4bb8-48b8-98ca-115a93e3c44e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:02.021509 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a2574190-f756-4f72-9aeb-e209b52b5af0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:02.031782 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a2574190-f756-4f72-9aeb-e209b52b5af0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:02.032465 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a2574190-f756-4f72-9aeb-e209b52b5af0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:02.222530 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-79ae6975-4bb8-48b8-98ca-115a93e3c44e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:02.226088 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a2574190-f756-4f72-9aeb-e209b52b5af0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:02.234068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a2574190-f756-4f72-9aeb-e209b52b5af0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.228616 Jul 05 10:11:02.234395 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-79ae6975-4bb8-48b8-98ca-115a93e3c44e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.226869 Jul 05 10:11:02.234633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 70/138] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:02 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:02.239222 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-acc5d72a-78aa-46b3-92c7-7c9c604dd79b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:02.239829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-acc5d72a-78aa-46b3-92c7-7c9c604dd79b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:02.244619 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 69/139] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:02 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:02.246546 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-acc5d72a-78aa-46b3-92c7-7c9c604dd79b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:02.247000 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-acc5d72a-78aa-46b3-92c7-7c9c604dd79b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:02.247764 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-acc5d72a-78aa-46b3-92c7-7c9c604dd79b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:02.248200 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-acc5d72a-78aa-46b3-92c7-7c9c604dd79b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:02.380039 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-acc5d72a-78aa-46b3-92c7-7c9c604dd79b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:02.387469 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-acc5d72a-78aa-46b3-92c7-7c9c604dd79b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.151735 Jul 05 10:11:02.388042 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 71/140] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:02 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:02.974265 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-4591b158-c3ce-4627-abfb-710da5728ad0 req-d1668873-50ad-4cd2-86bb-0750f005026b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f6c29e5f-2d52-4c7a-9122-11dbaf998421", "name": "network-vif-plugged", "status": "completed", "tag": "494e9e9b-34e3-4ca8-b312-38f821ddd851"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:11:02.983312 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-4591b158-c3ce-4627-abfb-710da5728ad0 req-d1668873-50ad-4cd2-86bb-0750f005026b service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:02.983888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-4591b158-c3ce-4627-abfb-710da5728ad0 req-d1668873-50ad-4cd2-86bb-0750f005026b service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:02.998386 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-4591b158-c3ce-4627-abfb-710da5728ad0 req-d1668873-50ad-4cd2-86bb-0750f005026b service nova] Creating event network-vif-plugged:494e9e9b-34e3-4ca8-b312-38f821ddd851 for instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 on hv-cinder-maste Jul 05 10:11:03.003519 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-4591b158-c3ce-4627-abfb-710da5728ad0 req-d1668873-50ad-4cd2-86bb-0750f005026b service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033243 Jul 05 10:11:03.004313 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 70/141] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:11:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:11:03.132817 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1510622916", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "networks": [{"uuid": "47de8f46-bb2e-4830-b096-bb218c860b15"}]}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:11:03.186637 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.network.neutron [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] validate_networks() for [('47de8f46-bb2e-4830-b096-bb218c860b15', None, None, None, None, None)] {{(pid=117813) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:11:03.253271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-72e67e36-ac7e-42f4-9db5-de306a8cdae7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:03.254105 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-72e67e36-ac7e-42f4-9db5-de306a8cdae7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:03.261158 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72e67e36-ac7e-42f4-9db5-de306a8cdae7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:03.261609 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72e67e36-ac7e-42f4-9db5-de306a8cdae7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:03.262403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72e67e36-ac7e-42f4-9db5-de306a8cdae7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:03.262807 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72e67e36-ac7e-42f4-9db5-de306a8cdae7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:03.469337 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-72e67e36-ac7e-42f4-9db5-de306a8cdae7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:03.477079 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-72e67e36-ac7e-42f4-9db5-de306a8cdae7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.229484 Jul 05 10:11:03.477904 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 71/142] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:03 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:03.482532 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-040f318f-05fb-4bda-9453-83db814a7b6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:03.483127 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-040f318f-05fb-4bda-9453-83db814a7b6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:03.488750 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-040f318f-05fb-4bda-9453-83db814a7b6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:03.489198 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-040f318f-05fb-4bda-9453-83db814a7b6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:03.489952 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-040f318f-05fb-4bda-9453-83db814a7b6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:03.490383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-040f318f-05fb-4bda-9453-83db814a7b6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:03.834291 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-040f318f-05fb-4bda-9453-83db814a7b6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:03.839245 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Flavor limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:11:03.839667 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Image limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:11:03.840136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Flavor pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:11:03.840582 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Image pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:11:03.841043 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:11:03.841635 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-040f318f-05fb-4bda-9453-83db814a7b6c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.362792 Jul 05 10:11:03.842213 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 72/143] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:03 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:03.850917 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:03.851356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:03.852136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:03.853174 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:03.853673 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-177fc79c-e731-4046-b720-c415562ffda1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:03.854292 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-177fc79c-e731-4046-b720-c415562ffda1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:03.860081 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-177fc79c-e731-4046-b720-c415562ffda1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:03.860573 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-177fc79c-e731-4046-b720-c415562ffda1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:03.861351 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-177fc79c-e731-4046-b720-c415562ffda1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:03.861787 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-177fc79c-e731-4046-b720-c415562ffda1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:03.870977 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Getting quotas for project c48f677354594fdc839367fff8b3dc29. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:11:03.875164 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Getting quotas for user d2d25be9c4674aa09b2975f1c6bc8c52 and project c48f677354594fdc839367fff8b3dc29. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:11:03.886300 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Going to run 1 instances... {{(pid=117813) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:11:03.888602 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117813) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:11:03.936060 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-321ef117-5b20-4bd4-9aff-ebe39b27442d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.807261 Jul 05 10:11:03.937085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 72/144] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:11:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 809 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:11:03.946508 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f54b8cd7-85cd-4b2b-8532-8a63daeac920 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:03.947165 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f54b8cd7-85cd-4b2b-8532-8a63daeac920 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:04.010203 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f54b8cd7-85cd-4b2b-8532-8a63daeac920 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:04.016932 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f54b8cd7-85cd-4b2b-8532-8a63daeac920 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1214 microversion: 2.1 time: 0.074298 Jul 05 10:11:04.017496 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 73/145] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:03 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1214 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:04.029813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-177fc79c-e731-4046-b720-c415562ffda1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:04.037145 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-177fc79c-e731-4046-b720-c415562ffda1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.187795 Jul 05 10:11:04.037711 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 73/146] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:03 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:04.493942 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b9d5a60e-ef69-420b-b856-ad7684b168e4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:04.494482 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b9d5a60e-ef69-420b-b856-ad7684b168e4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:04.501117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b9d5a60e-ef69-420b-b856-ad7684b168e4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:04.501574 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b9d5a60e-ef69-420b-b856-ad7684b168e4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:04.502336 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b9d5a60e-ef69-420b-b856-ad7684b168e4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:04.502775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b9d5a60e-ef69-420b-b856-ad7684b168e4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:04.630393 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b9d5a60e-ef69-420b-b856-ad7684b168e4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:04.638239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b9d5a60e-ef69-420b-b856-ad7684b168e4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.148148 Jul 05 10:11:04.638802 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 74/147] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:04 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:04.858071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0bd179c6-d1c6-4def-a587-3dd313234f6d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:04.858694 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0bd179c6-d1c6-4def-a587-3dd313234f6d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:04.864732 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0bd179c6-d1c6-4def-a587-3dd313234f6d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:04.865183 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0bd179c6-d1c6-4def-a587-3dd313234f6d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:04.865952 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0bd179c6-d1c6-4def-a587-3dd313234f6d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:04.866394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0bd179c6-d1c6-4def-a587-3dd313234f6d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:04.992300 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-0bd179c6-d1c6-4def-a587-3dd313234f6d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:05.000064 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0bd179c6-d1c6-4def-a587-3dd313234f6d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.145873 Jul 05 10:11:05.000652 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 74/148] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:04 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:05.034107 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9ff1c658-ca9d-4024-8816-36ce48708371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:05.034648 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9ff1c658-ca9d-4024-8816-36ce48708371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:05.040542 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9ff1c658-ca9d-4024-8816-36ce48708371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:05.040984 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9ff1c658-ca9d-4024-8816-36ce48708371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:05.041733 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9ff1c658-ca9d-4024-8816-36ce48708371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:05.042160 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9ff1c658-ca9d-4024-8816-36ce48708371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:05.054160 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c90f3949-61b4-4ba6-b192-b94b1620b7cc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:05.054771 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-c90f3949-61b4-4ba6-b192-b94b1620b7cc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:05.061278 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c90f3949-61b4-4ba6-b192-b94b1620b7cc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:05.061734 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c90f3949-61b4-4ba6-b192-b94b1620b7cc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:05.062492 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c90f3949-61b4-4ba6-b192-b94b1620b7cc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:05.062931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c90f3949-61b4-4ba6-b192-b94b1620b7cc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:05.104829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9ff1c658-ca9d-4024-8816-36ce48708371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:05.112177 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9ff1c658-ca9d-4024-8816-36ce48708371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1218 microversion: 2.1 time: 0.081865 Jul 05 10:11:05.112779 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 75/149] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:05 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1218 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:05.197864 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-c90f3949-61b4-4ba6-b192-b94b1620b7cc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:05.205508 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-c90f3949-61b4-4ba6-b192-b94b1620b7cc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.155144 Jul 05 10:11:05.206074 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 75/150] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:05 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:05.654939 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ca22342f-3203-4c89-8ed5-7d8580ff1a69 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:05.655457 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ca22342f-3203-4c89-8ed5-7d8580ff1a69 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:05.661371 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ca22342f-3203-4c89-8ed5-7d8580ff1a69 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:05.661812 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ca22342f-3203-4c89-8ed5-7d8580ff1a69 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:05.662556 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ca22342f-3203-4c89-8ed5-7d8580ff1a69 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:05.662982 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ca22342f-3203-4c89-8ed5-7d8580ff1a69 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:05.787970 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ca22342f-3203-4c89-8ed5-7d8580ff1a69 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:05.796146 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ca22342f-3203-4c89-8ed5-7d8580ff1a69 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.145033 Jul 05 10:11:05.796736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 76/151] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:05 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:06.017405 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-72990d53-b9aa-454a-bed0-a25397b88c61 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:06.018014 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-72990d53-b9aa-454a-bed0-a25397b88c61 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:06.024121 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72990d53-b9aa-454a-bed0-a25397b88c61 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:06.025180 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72990d53-b9aa-454a-bed0-a25397b88c61 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:06.025935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72990d53-b9aa-454a-bed0-a25397b88c61 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:06.026371 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72990d53-b9aa-454a-bed0-a25397b88c61 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:06.129519 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-df42b098-546a-402a-9e52-a433ec9a30cf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:06.130210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-df42b098-546a-402a-9e52-a433ec9a30cf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:06.136833 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-df42b098-546a-402a-9e52-a433ec9a30cf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:06.137288 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-df42b098-546a-402a-9e52-a433ec9a30cf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:06.138024 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-df42b098-546a-402a-9e52-a433ec9a30cf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:06.138453 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-df42b098-546a-402a-9e52-a433ec9a30cf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:06.161318 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-72990d53-b9aa-454a-bed0-a25397b88c61 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:06.168905 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-72990d53-b9aa-454a-bed0-a25397b88c61 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.155211 Jul 05 10:11:06.169475 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 76/152] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:06 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:06.197706 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-df42b098-546a-402a-9e52-a433ec9a30cf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:06.205136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-df42b098-546a-402a-9e52-a433ec9a30cf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1218 microversion: 2.1 time: 0.079423 Jul 05 10:11:06.205691 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 77/153] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:06 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1218 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:06.222019 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-da5bfa73-5f69-4151-b886-8c6c2e45b870 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:06.222618 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-da5bfa73-5f69-4151-b886-8c6c2e45b870 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:06.228542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-da5bfa73-5f69-4151-b886-8c6c2e45b870 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:06.228988 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-da5bfa73-5f69-4151-b886-8c6c2e45b870 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:06.229736 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-da5bfa73-5f69-4151-b886-8c6c2e45b870 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:06.230166 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-da5bfa73-5f69-4151-b886-8c6c2e45b870 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:06.354837 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-da5bfa73-5f69-4151-b886-8c6c2e45b870 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:06.362513 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-da5bfa73-5f69-4151-b886-8c6c2e45b870 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.144198 Jul 05 10:11:06.363073 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 77/154] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:06 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:06.812858 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-73cf78b7-5a40-4856-9fad-4802caaf75c7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:06.813470 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-73cf78b7-5a40-4856-9fad-4802caaf75c7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:06.819413 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-73cf78b7-5a40-4856-9fad-4802caaf75c7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:06.819867 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-73cf78b7-5a40-4856-9fad-4802caaf75c7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:06.820660 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-73cf78b7-5a40-4856-9fad-4802caaf75c7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:06.821102 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-73cf78b7-5a40-4856-9fad-4802caaf75c7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:06.952158 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-73cf78b7-5a40-4856-9fad-4802caaf75c7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:06.959534 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-73cf78b7-5a40-4856-9fad-4802caaf75c7 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.150540 Jul 05 10:11:06.960092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 78/155] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:06 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:07.185477 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a6353111-9d93-4341-bf35-91bd56b2b79b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:07.186064 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a6353111-9d93-4341-bf35-91bd56b2b79b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:07.191977 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a6353111-9d93-4341-bf35-91bd56b2b79b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:07.192441 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a6353111-9d93-4341-bf35-91bd56b2b79b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:07.193194 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a6353111-9d93-4341-bf35-91bd56b2b79b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:07.193622 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a6353111-9d93-4341-bf35-91bd56b2b79b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:07.222239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-4b5afdc6-111e-42ce-b364-dddc37ceb7fc tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:07.222868 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-4b5afdc6-111e-42ce-b364-dddc37ceb7fc tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:07.229429 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4b5afdc6-111e-42ce-b364-dddc37ceb7fc tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:07.229876 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4b5afdc6-111e-42ce-b364-dddc37ceb7fc tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:07.230639 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4b5afdc6-111e-42ce-b364-dddc37ceb7fc tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:07.231074 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4b5afdc6-111e-42ce-b364-dddc37ceb7fc tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:07.304707 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4b5afdc6-111e-42ce-b364-dddc37ceb7fc tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:07.312147 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-4b5afdc6-111e-42ce-b364-dddc37ceb7fc tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1218 microversion: 2.1 time: 0.093719 Jul 05 10:11:07.312753 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 79/156] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:07 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1218 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:07.340962 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a6353111-9d93-4341-bf35-91bd56b2b79b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:07.348185 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a6353111-9d93-4341-bf35-91bd56b2b79b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.166537 Jul 05 10:11:07.348782 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 78/157] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:07 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:07.379275 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c8f8a5ff-a3c5-44fb-9e4c-08baa69dcbcc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:07.379872 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c8f8a5ff-a3c5-44fb-9e4c-08baa69dcbcc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:07.386389 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c8f8a5ff-a3c5-44fb-9e4c-08baa69dcbcc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:07.386848 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c8f8a5ff-a3c5-44fb-9e4c-08baa69dcbcc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:07.387597 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c8f8a5ff-a3c5-44fb-9e4c-08baa69dcbcc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:07.388034 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c8f8a5ff-a3c5-44fb-9e4c-08baa69dcbcc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:07.517028 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c8f8a5ff-a3c5-44fb-9e4c-08baa69dcbcc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:07.524264 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c8f8a5ff-a3c5-44fb-9e4c-08baa69dcbcc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.148782 Jul 05 10:11:07.525448 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 80/158] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:07 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:07.976017 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-eb2b11c1-5f03-4714-8a3a-2ca2abce328b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:07.976687 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-eb2b11c1-5f03-4714-8a3a-2ca2abce328b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:07.982597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eb2b11c1-5f03-4714-8a3a-2ca2abce328b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:07.983041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eb2b11c1-5f03-4714-8a3a-2ca2abce328b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:07.983811 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eb2b11c1-5f03-4714-8a3a-2ca2abce328b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:07.984250 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eb2b11c1-5f03-4714-8a3a-2ca2abce328b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:08.111511 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-eb2b11c1-5f03-4714-8a3a-2ca2abce328b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:08.119452 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-eb2b11c1-5f03-4714-8a3a-2ca2abce328b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.147196 Jul 05 10:11:08.120014 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 79/159] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:07 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:08.329123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e3b4f3a5-ea2a-43da-8932-9826abb402f3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:08.329720 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e3b4f3a5-ea2a-43da-8932-9826abb402f3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:08.335617 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e3b4f3a5-ea2a-43da-8932-9826abb402f3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:08.336056 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e3b4f3a5-ea2a-43da-8932-9826abb402f3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:08.336842 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e3b4f3a5-ea2a-43da-8932-9826abb402f3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:08.337269 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e3b4f3a5-ea2a-43da-8932-9826abb402f3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:08.365445 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3256b9be-2410-4224-8610-42b14bd62b6f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:08.365982 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3256b9be-2410-4224-8610-42b14bd62b6f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:08.372899 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3256b9be-2410-4224-8610-42b14bd62b6f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:08.373356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3256b9be-2410-4224-8610-42b14bd62b6f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:08.374164 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3256b9be-2410-4224-8610-42b14bd62b6f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:08.374601 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3256b9be-2410-4224-8610-42b14bd62b6f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:08.393238 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=100 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:08.399686 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e3b4f3a5-ea2a-43da-8932-9826abb402f3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:08.407044 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e3b4f3a5-ea2a-43da-8932-9826abb402f3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1218 microversion: 2.1 time: 0.081743 Jul 05 10:11:08.407613 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 81/160] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:08 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1218 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:08.521272 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-3256b9be-2410-4224-8610-42b14bd62b6f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:08.529447 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3256b9be-2410-4224-8610-42b14bd62b6f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.167884 Jul 05 10:11:08.530014 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 80/161] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:08 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:08.542471 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1f456ea9-cf06-48ff-b530-4947a19dfe1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:08.543521 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1f456ea9-cf06-48ff-b530-4947a19dfe1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:08.549915 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1f456ea9-cf06-48ff-b530-4947a19dfe1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:08.550360 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1f456ea9-cf06-48ff-b530-4947a19dfe1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:08.551431 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1f456ea9-cf06-48ff-b530-4947a19dfe1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:08.551939 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1f456ea9-cf06-48ff-b530-4947a19dfe1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:08.693638 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1f456ea9-cf06-48ff-b530-4947a19dfe1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:08.697091 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=100 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:08.702041 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1f456ea9-cf06-48ff-b530-4947a19dfe1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.164061 Jul 05 10:11:08.702601 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 82/162] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:08 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:09.137508 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b71ba4b2-1ed7-4bab-9bab-5d019fa14768 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:09.138234 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b71ba4b2-1ed7-4bab-9bab-5d019fa14768 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:09.144233 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b71ba4b2-1ed7-4bab-9bab-5d019fa14768 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:09.144785 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b71ba4b2-1ed7-4bab-9bab-5d019fa14768 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:09.145613 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b71ba4b2-1ed7-4bab-9bab-5d019fa14768 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:09.146065 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b71ba4b2-1ed7-4bab-9bab-5d019fa14768 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:09.320055 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b71ba4b2-1ed7-4bab-9bab-5d019fa14768 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:09.328695 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b71ba4b2-1ed7-4bab-9bab-5d019fa14768 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.196427 Jul 05 10:11:09.329281 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 81/163] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:09 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:09.423530 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a1ac653d-ee4f-488d-9f6f-367e6aa06770 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:09.424134 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a1ac653d-ee4f-488d-9f6f-367e6aa06770 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:09.430333 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a1ac653d-ee4f-488d-9f6f-367e6aa06770 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:09.431323 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a1ac653d-ee4f-488d-9f6f-367e6aa06770 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:09.432074 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a1ac653d-ee4f-488d-9f6f-367e6aa06770 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:09.432551 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a1ac653d-ee4f-488d-9f6f-367e6aa06770 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:09.494457 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a1ac653d-ee4f-488d-9f6f-367e6aa06770 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:09.502656 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a1ac653d-ee4f-488d-9f6f-367e6aa06770 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1218 microversion: 2.1 time: 0.082834 Jul 05 10:11:09.503388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 83/164] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:09 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1218 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:09.549550 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2f786bdd-383b-4c97-976a-76455760afdc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:09.549550 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2f786bdd-383b-4c97-976a-76455760afdc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:09.554080 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2f786bdd-383b-4c97-976a-76455760afdc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:09.554695 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2f786bdd-383b-4c97-976a-76455760afdc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:09.556320 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2f786bdd-383b-4c97-976a-76455760afdc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:09.557129 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2f786bdd-383b-4c97-976a-76455760afdc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:09.723540 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8309da28-7608-4a53-b9c8-ccf73e3fc817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:09.724295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8309da28-7608-4a53-b9c8-ccf73e3fc817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:09.735293 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8309da28-7608-4a53-b9c8-ccf73e3fc817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:09.736136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8309da28-7608-4a53-b9c8-ccf73e3fc817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:09.737368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8309da28-7608-4a53-b9c8-ccf73e3fc817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:09.742683 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8309da28-7608-4a53-b9c8-ccf73e3fc817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:09.782076 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-2f786bdd-383b-4c97-976a-76455760afdc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:09.796556 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2f786bdd-383b-4c97-976a-76455760afdc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.252805 Jul 05 10:11:09.797409 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 82/165] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:09 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:09.950075 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8309da28-7608-4a53-b9c8-ccf73e3fc817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:09.957955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8309da28-7608-4a53-b9c8-ccf73e3fc817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.239724 Jul 05 10:11:09.958909 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 84/166] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:09 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:10.347108 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-eb80e41d-e427-40e7-b798-af5895f6157d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:10.347734 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-eb80e41d-e427-40e7-b798-af5895f6157d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:10.354701 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eb80e41d-e427-40e7-b798-af5895f6157d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:10.355164 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eb80e41d-e427-40e7-b798-af5895f6157d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:10.356119 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eb80e41d-e427-40e7-b798-af5895f6157d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:10.356686 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eb80e41d-e427-40e7-b798-af5895f6157d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:10.519648 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ab3ff285-ff62-49db-8bd7-20ea912ef276 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:10.520945 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ab3ff285-ff62-49db-8bd7-20ea912ef276 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:10.529620 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ab3ff285-ff62-49db-8bd7-20ea912ef276 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:10.530418 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ab3ff285-ff62-49db-8bd7-20ea912ef276 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:10.532049 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ab3ff285-ff62-49db-8bd7-20ea912ef276 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:10.532951 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ab3ff285-ff62-49db-8bd7-20ea912ef276 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:10.546348 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-eb80e41d-e427-40e7-b798-af5895f6157d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:10.557063 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-eb80e41d-e427-40e7-b798-af5895f6157d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.214994 Jul 05 10:11:10.558033 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 83/167] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:10 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:10.629327 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ab3ff285-ff62-49db-8bd7-20ea912ef276 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:10.636682 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ab3ff285-ff62-49db-8bd7-20ea912ef276 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1218 microversion: 2.1 time: 0.121546 Jul 05 10:11:10.637250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 85/168] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:10 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1218 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:10.814295 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d40fa9a0-f5e6-4e0a-a1f4-58b5e8c92f0a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:10.814961 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d40fa9a0-f5e6-4e0a-a1f4-58b5e8c92f0a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:10.824583 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d40fa9a0-f5e6-4e0a-a1f4-58b5e8c92f0a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:10.824583 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d40fa9a0-f5e6-4e0a-a1f4-58b5e8c92f0a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:10.824583 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d40fa9a0-f5e6-4e0a-a1f4-58b5e8c92f0a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:10.824583 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d40fa9a0-f5e6-4e0a-a1f4-58b5e8c92f0a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:10.959345 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d40fa9a0-f5e6-4e0a-a1f4-58b5e8c92f0a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:10.967390 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d40fa9a0-f5e6-4e0a-a1f4-58b5e8c92f0a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.156917 Jul 05 10:11:10.967951 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 84/169] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:10 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:10.975215 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-28c804a1-4a2e-49e4-b3b9-58f35a7e80ba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:10.976017 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-28c804a1-4a2e-49e4-b3b9-58f35a7e80ba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:10.983164 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-28c804a1-4a2e-49e4-b3b9-58f35a7e80ba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:10.983612 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-28c804a1-4a2e-49e4-b3b9-58f35a7e80ba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:10.984533 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-28c804a1-4a2e-49e4-b3b9-58f35a7e80ba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:10.984972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-28c804a1-4a2e-49e4-b3b9-58f35a7e80ba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:11.135083 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-28c804a1-4a2e-49e4-b3b9-58f35a7e80ba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:11.143831 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-28c804a1-4a2e-49e4-b3b9-58f35a7e80ba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.172641 Jul 05 10:11:11.144431 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 86/170] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:10 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:11.575771 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-69ef13d6-d64a-4c33-8b4c-200430cb226d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:11.576446 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-69ef13d6-d64a-4c33-8b4c-200430cb226d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:11.582406 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-69ef13d6-d64a-4c33-8b4c-200430cb226d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:11.582907 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-69ef13d6-d64a-4c33-8b4c-200430cb226d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:11.583583 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-69ef13d6-d64a-4c33-8b4c-200430cb226d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:11.584117 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-69ef13d6-d64a-4c33-8b4c-200430cb226d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:11.653055 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-80b1148b-cc82-4363-b3a5-af8fccdbdd69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:11.654013 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-80b1148b-cc82-4363-b3a5-af8fccdbdd69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:11.660244 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80b1148b-cc82-4363-b3a5-af8fccdbdd69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:11.660786 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80b1148b-cc82-4363-b3a5-af8fccdbdd69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:11.661601 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80b1148b-cc82-4363-b3a5-af8fccdbdd69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:11.662478 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80b1148b-cc82-4363-b3a5-af8fccdbdd69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:11.744246 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-80b1148b-cc82-4363-b3a5-af8fccdbdd69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:11.751823 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-69ef13d6-d64a-4c33-8b4c-200430cb226d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:11.752111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-80b1148b-cc82-4363-b3a5-af8fccdbdd69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1218 microversion: 2.1 time: 0.102770 Jul 05 10:11:11.752792 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 87/171] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:11 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1218 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:11.761078 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-69ef13d6-d64a-4c33-8b4c-200430cb226d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.189998 Jul 05 10:11:11.761642 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 85/172] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:11 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:11.983463 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c9577c2b-8137-4a6c-9da5-a77f4abee09e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:11.984066 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c9577c2b-8137-4a6c-9da5-a77f4abee09e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:11.989884 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c9577c2b-8137-4a6c-9da5-a77f4abee09e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:11.990329 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c9577c2b-8137-4a6c-9da5-a77f4abee09e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:11.991083 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c9577c2b-8137-4a6c-9da5-a77f4abee09e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:11.991512 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c9577c2b-8137-4a6c-9da5-a77f4abee09e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:12.124846 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c9577c2b-8137-4a6c-9da5-a77f4abee09e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:12.132441 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c9577c2b-8137-4a6c-9da5-a77f4abee09e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.152835 Jul 05 10:11:12.133016 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 88/173] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:11 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:12.161213 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-127f183e-1081-4b3b-9c6f-9402c11cfda3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:12.161824 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-127f183e-1081-4b3b-9c6f-9402c11cfda3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:12.168502 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-127f183e-1081-4b3b-9c6f-9402c11cfda3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:12.168953 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-127f183e-1081-4b3b-9c6f-9402c11cfda3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:12.169776 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-127f183e-1081-4b3b-9c6f-9402c11cfda3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:12.170214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-127f183e-1081-4b3b-9c6f-9402c11cfda3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:12.298794 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-127f183e-1081-4b3b-9c6f-9402c11cfda3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:12.307086 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-127f183e-1081-4b3b-9c6f-9402c11cfda3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.149470 Jul 05 10:11:12.307710 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 86/174] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:12 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:12.769341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-06d25400-db32-4425-8015-7f4a650aaed6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:12.769955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-06d25400-db32-4425-8015-7f4a650aaed6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:12.776763 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-06d25400-db32-4425-8015-7f4a650aaed6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:12.777215 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-06d25400-db32-4425-8015-7f4a650aaed6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:12.777977 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-06d25400-db32-4425-8015-7f4a650aaed6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:12.778406 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-737e117e-a369-4f6c-add6-94e42a8ffc2c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:12.778804 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-06d25400-db32-4425-8015-7f4a650aaed6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:12.779156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-737e117e-a369-4f6c-add6-94e42a8ffc2c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:12.785167 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-737e117e-a369-4f6c-add6-94e42a8ffc2c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:12.785612 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-737e117e-a369-4f6c-add6-94e42a8ffc2c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:12.786362 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-737e117e-a369-4f6c-add6-94e42a8ffc2c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:12.786791 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-737e117e-a369-4f6c-add6-94e42a8ffc2c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:12.864135 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-06d25400-db32-4425-8015-7f4a650aaed6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:12.871466 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-06d25400-db32-4425-8015-7f4a650aaed6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1218 microversion: 2.1 time: 0.105932 Jul 05 10:11:12.872032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 89/175] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:12 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1218 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:12.934868 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-737e117e-a369-4f6c-add6-94e42a8ffc2c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:12.943142 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-737e117e-a369-4f6c-add6-94e42a8ffc2c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.169210 Jul 05 10:11:12.943881 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 87/176] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:12 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:13.149064 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d04ef5f8-5893-4293-baf9-a60a60070a70 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:13.149719 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d04ef5f8-5893-4293-baf9-a60a60070a70 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:13.155890 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d04ef5f8-5893-4293-baf9-a60a60070a70 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:13.156337 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d04ef5f8-5893-4293-baf9-a60a60070a70 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:13.157126 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d04ef5f8-5893-4293-baf9-a60a60070a70 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:13.157559 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d04ef5f8-5893-4293-baf9-a60a60070a70 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:13.283089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d04ef5f8-5893-4293-baf9-a60a60070a70 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:13.290392 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d04ef5f8-5893-4293-baf9-a60a60070a70 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.145102 Jul 05 10:11:13.290959 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 90/177] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:13 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:13.325199 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7f890e1d-8fc2-48c5-9366-dcf7e65fd542 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:13.325800 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7f890e1d-8fc2-48c5-9366-dcf7e65fd542 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:13.331751 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7f890e1d-8fc2-48c5-9366-dcf7e65fd542 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:13.332194 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7f890e1d-8fc2-48c5-9366-dcf7e65fd542 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:13.333200 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7f890e1d-8fc2-48c5-9366-dcf7e65fd542 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:13.333634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7f890e1d-8fc2-48c5-9366-dcf7e65fd542 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:13.463596 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-7f890e1d-8fc2-48c5-9366-dcf7e65fd542 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:13.471072 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7f890e1d-8fc2-48c5-9366-dcf7e65fd542 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.150595 Jul 05 10:11:13.471724 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 88/178] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:13 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:13.887925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-80b724aa-0e6e-4e5c-8000-f597e2b1601e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:13.888582 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-80b724aa-0e6e-4e5c-8000-f597e2b1601e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:13.890265 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:13.895873 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80b724aa-0e6e-4e5c-8000-f597e2b1601e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:13.896549 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80b724aa-0e6e-4e5c-8000-f597e2b1601e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:13.897311 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80b724aa-0e6e-4e5c-8000-f597e2b1601e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:13.897807 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80b724aa-0e6e-4e5c-8000-f597e2b1601e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:13.961623 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-18633d67-2441-444b-a01d-649539514ed9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:13.963604 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-18633d67-2441-444b-a01d-649539514ed9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:13.964025 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-80b724aa-0e6e-4e5c-8000-f597e2b1601e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:13.970457 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-18633d67-2441-444b-a01d-649539514ed9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:13.971236 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-18633d67-2441-444b-a01d-649539514ed9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:13.972227 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-18633d67-2441-444b-a01d-649539514ed9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:13.972855 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-80b724aa-0e6e-4e5c-8000-f597e2b1601e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1218 microversion: 2.1 time: 0.088621 Jul 05 10:11:13.973086 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-18633d67-2441-444b-a01d-649539514ed9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:13.973439 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 91/179] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:13 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1218 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:14.119272 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-18633d67-2441-444b-a01d-649539514ed9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:14.126639 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-18633d67-2441-444b-a01d-649539514ed9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.168983 Jul 05 10:11:14.127203 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 89/180] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:13 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:14.306916 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-120aeed5-e108-4916-9236-e61a4ed6f9a2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:14.307513 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-120aeed5-e108-4916-9236-e61a4ed6f9a2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:14.309345 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=8 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:14.309971 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:14.315613 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-120aeed5-e108-4916-9236-e61a4ed6f9a2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:14.316051 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-120aeed5-e108-4916-9236-e61a4ed6f9a2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:14.316833 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-120aeed5-e108-4916-9236-e61a4ed6f9a2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:14.317259 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-120aeed5-e108-4916-9236-e61a4ed6f9a2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:14.442040 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-120aeed5-e108-4916-9236-e61a4ed6f9a2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:14.450215 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-120aeed5-e108-4916-9236-e61a4ed6f9a2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.147126 Jul 05 10:11:14.450785 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 92/181] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:14 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:14.487942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-028dc3ff-5eb8-4157-aa9f-ca84edacf423 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:14.488673 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-028dc3ff-5eb8-4157-aa9f-ca84edacf423 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:14.490535 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=88 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:14.496487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-028dc3ff-5eb8-4157-aa9f-ca84edacf423 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:14.496947 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-028dc3ff-5eb8-4157-aa9f-ca84edacf423 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:14.497760 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-028dc3ff-5eb8-4157-aa9f-ca84edacf423 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:14.498196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-028dc3ff-5eb8-4157-aa9f-ca84edacf423 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:14.636482 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-028dc3ff-5eb8-4157-aa9f-ca84edacf423 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:14.644735 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-028dc3ff-5eb8-4157-aa9f-ca84edacf423 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.160416 Jul 05 10:11:14.654133 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 90/182] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:14 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:14.990302 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-189526f3-2442-40ec-8b84-9521ca336c86 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:14.991026 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-189526f3-2442-40ec-8b84-9521ca336c86 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:14.999043 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-189526f3-2442-40ec-8b84-9521ca336c86 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:14.999920 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-189526f3-2442-40ec-8b84-9521ca336c86 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:15.001055 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-189526f3-2442-40ec-8b84-9521ca336c86 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:15.001698 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-189526f3-2442-40ec-8b84-9521ca336c86 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:15.106123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-189526f3-2442-40ec-8b84-9521ca336c86 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:15.117993 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-189526f3-2442-40ec-8b84-9521ca336c86 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1218 microversion: 2.1 time: 0.131171 Jul 05 10:11:15.119672 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 93/183] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:14 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1218 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:15.144261 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ec1824d9-2079-4b04-b9d8-be9657e5eb84 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:15.145479 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ec1824d9-2079-4b04-b9d8-be9657e5eb84 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:15.151712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ec1824d9-2079-4b04-b9d8-be9657e5eb84 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:15.152165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ec1824d9-2079-4b04-b9d8-be9657e5eb84 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:15.152974 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ec1824d9-2079-4b04-b9d8-be9657e5eb84 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:15.153411 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ec1824d9-2079-4b04-b9d8-be9657e5eb84 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:15.302045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ec1824d9-2079-4b04-b9d8-be9657e5eb84 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:15.310370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ec1824d9-2079-4b04-b9d8-be9657e5eb84 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.170580 Jul 05 10:11:15.310936 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 91/184] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:15 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:15.467136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-72de1522-0fb2-4af8-8a28-b520cefd0c78 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:15.467742 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-72de1522-0fb2-4af8-8a28-b520cefd0c78 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:15.473841 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-72de1522-0fb2-4af8-8a28-b520cefd0c78 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:15.474315 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-72de1522-0fb2-4af8-8a28-b520cefd0c78 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:15.475117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-72de1522-0fb2-4af8-8a28-b520cefd0c78 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:15.475589 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-72de1522-0fb2-4af8-8a28-b520cefd0c78 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:15.610043 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-72de1522-0fb2-4af8-8a28-b520cefd0c78 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:15.617334 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-72de1522-0fb2-4af8-8a28-b520cefd0c78 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.153988 Jul 05 10:11:15.617895 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 94/185] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:15 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:15.661402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-dbf16460-234a-40de-adc9-644f2e5d5612 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:15.662044 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-dbf16460-234a-40de-adc9-644f2e5d5612 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:15.668124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dbf16460-234a-40de-adc9-644f2e5d5612 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:15.668607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dbf16460-234a-40de-adc9-644f2e5d5612 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:15.669405 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dbf16460-234a-40de-adc9-644f2e5d5612 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:15.669939 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dbf16460-234a-40de-adc9-644f2e5d5612 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:15.803158 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-dbf16460-234a-40de-adc9-644f2e5d5612 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:15.812804 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-dbf16460-234a-40de-adc9-644f2e5d5612 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.155133 Jul 05 10:11:15.813391 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 92/186] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:15 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:16.138913 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ac0b1b66-bf4d-4657-ab98-42defc64dad4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:16.140093 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ac0b1b66-bf4d-4657-ab98-42defc64dad4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:16.147110 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ac0b1b66-bf4d-4657-ab98-42defc64dad4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:16.147822 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ac0b1b66-bf4d-4657-ab98-42defc64dad4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:16.148622 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ac0b1b66-bf4d-4657-ab98-42defc64dad4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:16.149386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ac0b1b66-bf4d-4657-ab98-42defc64dad4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:16.222689 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ac0b1b66-bf4d-4657-ab98-42defc64dad4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:16.231550 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ac0b1b66-bf4d-4657-ab98-42defc64dad4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1210 microversion: 2.1 time: 0.097795 Jul 05 10:11:16.232466 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 95/187] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:16 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1210 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:16.330239 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2c12e11b-f0c3-415a-b3cf-d5ba3e17c485 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:16.331143 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2c12e11b-f0c3-415a-b3cf-d5ba3e17c485 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:16.342727 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2c12e11b-f0c3-415a-b3cf-d5ba3e17c485 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:16.343431 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2c12e11b-f0c3-415a-b3cf-d5ba3e17c485 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:16.347674 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2c12e11b-f0c3-415a-b3cf-d5ba3e17c485 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:16.348546 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2c12e11b-f0c3-415a-b3cf-d5ba3e17c485 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:16.563566 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-2c12e11b-f0c3-415a-b3cf-d5ba3e17c485 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:16.576200 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2c12e11b-f0c3-415a-b3cf-d5ba3e17c485 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.251528 Jul 05 10:11:16.578735 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 93/188] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:16 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:16.621386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-7efd062a-045f-4b15-a477-346e252e03da req-94dc769f-b0f4-45d8-b697-96cd9b6cacca service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fc065612-7f7d-45fb-89cc-9f332fb15dd1", "name": "network-vif-plugged", "status": "completed", "tag": "ff58dc45-3b6a-4ad9-a225-3ec518174c19"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:11:16.632333 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-7efd062a-045f-4b15-a477-346e252e03da req-94dc769f-b0f4-45d8-b697-96cd9b6cacca service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:16.632874 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-7efd062a-045f-4b15-a477-346e252e03da req-94dc769f-b0f4-45d8-b697-96cd9b6cacca service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:16.635729 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-985ebce7-6882-431b-8535-c0f73a5c109b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:16.636577 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-985ebce7-6882-431b-8535-c0f73a5c109b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:16.648600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-7efd062a-045f-4b15-a477-346e252e03da req-94dc769f-b0f4-45d8-b697-96cd9b6cacca service nova] Creating event network-vif-plugged:ff58dc45-3b6a-4ad9-a225-3ec518174c19 for instance fc065612-7f7d-45fb-89cc-9f332fb15dd1 on hv-cinder-maste Jul 05 10:11:16.649459 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-985ebce7-6882-431b-8535-c0f73a5c109b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:16.650012 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-985ebce7-6882-431b-8535-c0f73a5c109b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:16.654593 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-985ebce7-6882-431b-8535-c0f73a5c109b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:16.655161 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-985ebce7-6882-431b-8535-c0f73a5c109b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:16.656730 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-7efd062a-045f-4b15-a477-346e252e03da req-94dc769f-b0f4-45d8-b697-96cd9b6cacca service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039490 Jul 05 10:11:16.657529 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 96/189] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:11:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:11:16.830953 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9d30dd5c-c38d-4ccc-a429-ed863eb344ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:16.831858 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9d30dd5c-c38d-4ccc-a429-ed863eb344ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:16.840887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9d30dd5c-c38d-4ccc-a429-ed863eb344ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:16.841593 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9d30dd5c-c38d-4ccc-a429-ed863eb344ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:16.842364 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9d30dd5c-c38d-4ccc-a429-ed863eb344ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:16.842832 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9d30dd5c-c38d-4ccc-a429-ed863eb344ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:16.959177 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-985ebce7-6882-431b-8535-c0f73a5c109b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:16.970028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-985ebce7-6882-431b-8535-c0f73a5c109b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.338531 Jul 05 10:11:16.971490 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 94/190] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:16 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:17.127634 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9d30dd5c-c38d-4ccc-a429-ed863eb344ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:17.137088 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9d30dd5c-c38d-4ccc-a429-ed863eb344ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.311286 Jul 05 10:11:17.137758 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 97/191] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:16 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:17.250179 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-81dcb01c-eade-49f0-9f38-0f19a97bd9f6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:17.251045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-81dcb01c-eade-49f0-9f38-0f19a97bd9f6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:17.257515 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-81dcb01c-eade-49f0-9f38-0f19a97bd9f6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:17.258037 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-81dcb01c-eade-49f0-9f38-0f19a97bd9f6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:17.259528 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-81dcb01c-eade-49f0-9f38-0f19a97bd9f6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:17.260070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-81dcb01c-eade-49f0-9f38-0f19a97bd9f6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:17.340842 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-81dcb01c-eade-49f0-9f38-0f19a97bd9f6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:17.350339 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-81dcb01c-eade-49f0-9f38-0f19a97bd9f6 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1266 microversion: 2.1 time: 0.104515 Jul 05 10:11:17.351160 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 95/192] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:17 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1266 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:17.595199 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-eb745c8b-8827-4f3a-b03b-fe0d09d41f8b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:17.595776 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-eb745c8b-8827-4f3a-b03b-fe0d09d41f8b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:17.603822 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-eb745c8b-8827-4f3a-b03b-fe0d09d41f8b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:17.604322 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-eb745c8b-8827-4f3a-b03b-fe0d09d41f8b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:17.605173 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-eb745c8b-8827-4f3a-b03b-fe0d09d41f8b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:17.605712 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-eb745c8b-8827-4f3a-b03b-fe0d09d41f8b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:17.748233 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-eb745c8b-8827-4f3a-b03b-fe0d09d41f8b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:17.757108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-eb745c8b-8827-4f3a-b03b-fe0d09d41f8b tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.166131 Jul 05 10:11:17.757786 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 98/193] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:17 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:17.988062 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-626a2210-9e4b-4c10-9905-e15400c547f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:17.988709 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-626a2210-9e4b-4c10-9905-e15400c547f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:17.995385 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-626a2210-9e4b-4c10-9905-e15400c547f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:17.995837 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-626a2210-9e4b-4c10-9905-e15400c547f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:17.996629 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-626a2210-9e4b-4c10-9905-e15400c547f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:17.997070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-626a2210-9e4b-4c10-9905-e15400c547f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:18.129912 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-626a2210-9e4b-4c10-9905-e15400c547f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:18.137325 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-626a2210-9e4b-4c10-9905-e15400c547f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.153198 Jul 05 10:11:18.137888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 96/194] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:17 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:18.153729 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2346d206-4418-4df3-990f-7708980c312c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:18.154338 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2346d206-4418-4df3-990f-7708980c312c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:18.160539 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2346d206-4418-4df3-990f-7708980c312c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:18.161174 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2346d206-4418-4df3-990f-7708980c312c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:18.161941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2346d206-4418-4df3-990f-7708980c312c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:18.162383 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2346d206-4418-4df3-990f-7708980c312c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:18.291337 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2346d206-4418-4df3-990f-7708980c312c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:18.299022 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2346d206-4418-4df3-990f-7708980c312c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.149143 Jul 05 10:11:18.299584 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 99/195] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:18 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:18.368151 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-21610539-d0ad-4e5f-bce5-d891885a1d8e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:18.368857 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-21610539-d0ad-4e5f-bce5-d891885a1d8e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:18.374715 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-21610539-d0ad-4e5f-bce5-d891885a1d8e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:18.375163 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-21610539-d0ad-4e5f-bce5-d891885a1d8e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:18.375927 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-21610539-d0ad-4e5f-bce5-d891885a1d8e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:18.376367 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-21610539-d0ad-4e5f-bce5-d891885a1d8e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:18.380401 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=100 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:18.442045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-21610539-d0ad-4e5f-bce5-d891885a1d8e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:18.449797 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-21610539-d0ad-4e5f-bce5-d891885a1d8e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.085286 Jul 05 10:11:18.450610 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 97/196] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:18 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:18.774684 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-74f23964-0c3f-4171-b1c9-3f201016fe00 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:18.775884 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-74f23964-0c3f-4171-b1c9-3f201016fe00 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:18.782065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-74f23964-0c3f-4171-b1c9-3f201016fe00 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:18.782503 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-74f23964-0c3f-4171-b1c9-3f201016fe00 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:18.783248 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-74f23964-0c3f-4171-b1c9-3f201016fe00 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:18.783674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-74f23964-0c3f-4171-b1c9-3f201016fe00 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:18.787765 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=100 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:18.944296 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-74f23964-0c3f-4171-b1c9-3f201016fe00 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:18.953949 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-74f23964-0c3f-4171-b1c9-3f201016fe00 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.183012 Jul 05 10:11:18.954599 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 100/197] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:18 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:19.153820 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-498d4036-2f1c-4b83-8f3d-938363501f45 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:19.154437 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-498d4036-2f1c-4b83-8f3d-938363501f45 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:19.160315 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-498d4036-2f1c-4b83-8f3d-938363501f45 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:19.160802 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-498d4036-2f1c-4b83-8f3d-938363501f45 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:19.161558 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-498d4036-2f1c-4b83-8f3d-938363501f45 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:19.161993 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-498d4036-2f1c-4b83-8f3d-938363501f45 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:19.300780 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-498d4036-2f1c-4b83-8f3d-938363501f45 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:19.325294 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-498d4036-2f1c-4b83-8f3d-938363501f45 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.174930 Jul 05 10:11:19.325949 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1bdf00d4-9bf7-4539-85af-5d87e0019a7a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:19.326062 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 98/198] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:19 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:19.326949 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1bdf00d4-9bf7-4539-85af-5d87e0019a7a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:19.335717 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1bdf00d4-9bf7-4539-85af-5d87e0019a7a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:19.336240 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1bdf00d4-9bf7-4539-85af-5d87e0019a7a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:19.337133 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1bdf00d4-9bf7-4539-85af-5d87e0019a7a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:19.337627 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1bdf00d4-9bf7-4539-85af-5d87e0019a7a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:19.466714 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0857d3b7-5eff-421f-a6d2-7ede2f50a56d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:19.467495 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0857d3b7-5eff-421f-a6d2-7ede2f50a56d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:19.473801 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0857d3b7-5eff-421f-a6d2-7ede2f50a56d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:19.474248 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0857d3b7-5eff-421f-a6d2-7ede2f50a56d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:19.475029 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0857d3b7-5eff-421f-a6d2-7ede2f50a56d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:19.476252 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0857d3b7-5eff-421f-a6d2-7ede2f50a56d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:19.653215 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1bdf00d4-9bf7-4539-85af-5d87e0019a7a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:19.656131 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-0857d3b7-5eff-421f-a6d2-7ede2f50a56d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:19.662291 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1bdf00d4-9bf7-4539-85af-5d87e0019a7a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.350980 Jul 05 10:11:19.663091 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 101/199] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:19 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:19.670195 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0857d3b7-5eff-421f-a6d2-7ede2f50a56d tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.201488 Jul 05 10:11:19.670195 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 99/200] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:19 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:19.971030 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7255ba41-5ab8-4ca4-96af-f3f468a085ab tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:19.971675 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7255ba41-5ab8-4ca4-96af-f3f468a085ab tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:19.979452 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7255ba41-5ab8-4ca4-96af-f3f468a085ab tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:19.979899 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7255ba41-5ab8-4ca4-96af-f3f468a085ab tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:19.980681 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7255ba41-5ab8-4ca4-96af-f3f468a085ab tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:19.981116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7255ba41-5ab8-4ca4-96af-f3f468a085ab tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:20.108598 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-7255ba41-5ab8-4ca4-96af-f3f468a085ab tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:20.116187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7255ba41-5ab8-4ca4-96af-f3f468a085ab tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.148883 Jul 05 10:11:20.116863 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 100/201] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:19 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:20.343590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-760dab81-ff9b-4342-8b76-d03ca49b0005 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:20.344187 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-760dab81-ff9b-4342-8b76-d03ca49b0005 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:20.350734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-760dab81-ff9b-4342-8b76-d03ca49b0005 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:20.351176 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-760dab81-ff9b-4342-8b76-d03ca49b0005 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:20.351921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-760dab81-ff9b-4342-8b76-d03ca49b0005 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:20.352350 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-760dab81-ff9b-4342-8b76-d03ca49b0005 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:20.485802 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-760dab81-ff9b-4342-8b76-d03ca49b0005 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:20.494396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-760dab81-ff9b-4342-8b76-d03ca49b0005 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.154629 Jul 05 10:11:20.494959 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 102/202] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:20 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:20.680097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-422b8a32-228e-4798-988c-9288be60fdba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:20.680727 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-422b8a32-228e-4798-988c-9288be60fdba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:20.680925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-fe0aa404-bc94-4d2e-acbb-9256d63b0d33 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:20.681567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-fe0aa404-bc94-4d2e-acbb-9256d63b0d33 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:20.686683 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-422b8a32-228e-4798-988c-9288be60fdba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:20.687129 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-422b8a32-228e-4798-988c-9288be60fdba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:20.687416 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fe0aa404-bc94-4d2e-acbb-9256d63b0d33 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:20.687877 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-422b8a32-228e-4798-988c-9288be60fdba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:20.688137 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fe0aa404-bc94-4d2e-acbb-9256d63b0d33 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:20.688307 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-422b8a32-228e-4798-988c-9288be60fdba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:20.688925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fe0aa404-bc94-4d2e-acbb-9256d63b0d33 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:20.689361 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fe0aa404-bc94-4d2e-acbb-9256d63b0d33 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:20.775770 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-fe0aa404-bc94-4d2e-acbb-9256d63b0d33 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:20.783370 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-fe0aa404-bc94-4d2e-acbb-9256d63b0d33 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.106214 Jul 05 10:11:20.783934 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 103/203] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:20 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:20.841787 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-422b8a32-228e-4798-988c-9288be60fdba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:20.849088 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-422b8a32-228e-4798-988c-9288be60fdba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.172977 Jul 05 10:11:20.849673 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 101/204] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:20 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:21.131799 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c58db903-0c90-4af4-9532-3cdecbc6263f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:21.132434 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c58db903-0c90-4af4-9532-3cdecbc6263f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:21.138910 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c58db903-0c90-4af4-9532-3cdecbc6263f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:21.139356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c58db903-0c90-4af4-9532-3cdecbc6263f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:21.140121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c58db903-0c90-4af4-9532-3cdecbc6263f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:21.140595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c58db903-0c90-4af4-9532-3cdecbc6263f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:21.267010 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c58db903-0c90-4af4-9532-3cdecbc6263f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:21.274346 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c58db903-0c90-4af4-9532-3cdecbc6263f tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.146327 Jul 05 10:11:21.274920 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 104/205] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:21 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:21.511249 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3174af6e-f4b7-4144-910f-459e6d0fb4ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:21.511866 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3174af6e-f4b7-4144-910f-459e6d0fb4ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:21.517907 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3174af6e-f4b7-4144-910f-459e6d0fb4ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:21.518414 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3174af6e-f4b7-4144-910f-459e6d0fb4ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:21.519175 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3174af6e-f4b7-4144-910f-459e6d0fb4ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:21.519608 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3174af6e-f4b7-4144-910f-459e6d0fb4ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:21.645098 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-3174af6e-f4b7-4144-910f-459e6d0fb4ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:21.652329 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3174af6e-f4b7-4144-910f-459e6d0fb4ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.145419 Jul 05 10:11:21.652948 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 102/206] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:21 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:21.800276 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-de570329-4d3e-4bee-af15-c8d7843aa583 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:21.800911 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-de570329-4d3e-4bee-af15-c8d7843aa583 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:21.807700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-de570329-4d3e-4bee-af15-c8d7843aa583 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:21.808136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-de570329-4d3e-4bee-af15-c8d7843aa583 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:21.808909 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-de570329-4d3e-4bee-af15-c8d7843aa583 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:21.809330 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-de570329-4d3e-4bee-af15-c8d7843aa583 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:21.870562 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-bbcae171-8e21-4371-ac77-b9bcef41564d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:21.871270 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bbcae171-8e21-4371-ac77-b9bcef41564d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:21.877420 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bbcae171-8e21-4371-ac77-b9bcef41564d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:21.877870 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bbcae171-8e21-4371-ac77-b9bcef41564d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:21.878636 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bbcae171-8e21-4371-ac77-b9bcef41564d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:21.879068 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bbcae171-8e21-4371-ac77-b9bcef41564d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:21.879439 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-de570329-4d3e-4bee-af15-c8d7843aa583 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:21.886722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-de570329-4d3e-4bee-af15-c8d7843aa583 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.090279 Jul 05 10:11:21.887282 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 105/207] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:21 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:22.010047 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-bbcae171-8e21-4371-ac77-b9bcef41564d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:22.017412 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-bbcae171-8e21-4371-ac77-b9bcef41564d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.154417 Jul 05 10:11:22.017975 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 103/208] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:21 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:22.291313 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-153c2025-8ac6-434c-8f8f-144be52bc5e5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:22.292326 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-153c2025-8ac6-434c-8f8f-144be52bc5e5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:22.298203 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-153c2025-8ac6-434c-8f8f-144be52bc5e5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:22.298648 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-153c2025-8ac6-434c-8f8f-144be52bc5e5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:22.299399 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-153c2025-8ac6-434c-8f8f-144be52bc5e5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:22.299832 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-153c2025-8ac6-434c-8f8f-144be52bc5e5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:22.427432 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-153c2025-8ac6-434c-8f8f-144be52bc5e5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:22.434792 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-153c2025-8ac6-434c-8f8f-144be52bc5e5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1315 microversion: 2.1 time: 0.147746 Jul 05 10:11:22.435529 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 106/209] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:22 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1315 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:22.669269 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-1ebe86f2-b591-4c89-b25f-45016715c90d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:22.669800 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-1ebe86f2-b591-4c89-b25f-45016715c90d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:22.675779 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1ebe86f2-b591-4c89-b25f-45016715c90d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:22.676222 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1ebe86f2-b591-4c89-b25f-45016715c90d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:22.677015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1ebe86f2-b591-4c89-b25f-45016715c90d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:22.677453 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1ebe86f2-b591-4c89-b25f-45016715c90d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:22.815462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1ebe86f2-b591-4c89-b25f-45016715c90d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:22.823426 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-1ebe86f2-b591-4c89-b25f-45016715c90d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.158155 Jul 05 10:11:22.823997 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 104/210] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:22 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:22.903025 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b2e6600c-1b59-4262-9d61-86837bc41811 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:22.903634 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b2e6600c-1b59-4262-9d61-86837bc41811 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:22.909618 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b2e6600c-1b59-4262-9d61-86837bc41811 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:22.910073 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b2e6600c-1b59-4262-9d61-86837bc41811 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:22.910838 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b2e6600c-1b59-4262-9d61-86837bc41811 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:22.911276 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b2e6600c-1b59-4262-9d61-86837bc41811 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:22.978926 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b2e6600c-1b59-4262-9d61-86837bc41811 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:22.986578 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b2e6600c-1b59-4262-9d61-86837bc41811 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.087279 Jul 05 10:11:22.987239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 107/211] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:22 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:23.033796 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-46aa1eb9-8d44-4b42-a30c-efe611ad7c89 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:23.034458 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-46aa1eb9-8d44-4b42-a30c-efe611ad7c89 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:23.040771 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-46aa1eb9-8d44-4b42-a30c-efe611ad7c89 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:23.041223 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-46aa1eb9-8d44-4b42-a30c-efe611ad7c89 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:23.041990 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-46aa1eb9-8d44-4b42-a30c-efe611ad7c89 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:23.042430 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-46aa1eb9-8d44-4b42-a30c-efe611ad7c89 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:23.171718 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-46aa1eb9-8d44-4b42-a30c-efe611ad7c89 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:23.179085 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-46aa1eb9-8d44-4b42-a30c-efe611ad7c89 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.149200 Jul 05 10:11:23.179651 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 105/212] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:23 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:23.452220 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-29b596b3-1c17-41a6-a0e9-44ff7fc94b2e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:23.452781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-29b596b3-1c17-41a6-a0e9-44ff7fc94b2e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:23.458680 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-29b596b3-1c17-41a6-a0e9-44ff7fc94b2e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:23.459120 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-29b596b3-1c17-41a6-a0e9-44ff7fc94b2e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:23.459866 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-29b596b3-1c17-41a6-a0e9-44ff7fc94b2e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:23.460297 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-29b596b3-1c17-41a6-a0e9-44ff7fc94b2e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:23.597382 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-29b596b3-1c17-41a6-a0e9-44ff7fc94b2e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:23.604977 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-29b596b3-1c17-41a6-a0e9-44ff7fc94b2e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1502 microversion: 2.1 time: 0.156681 Jul 05 10:11:23.605782 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 108/213] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:23 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1502 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:23.849325 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-9273af77-854f-4cf7-9f71-dd53fd1e8772 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:23.849325 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-9273af77-854f-4cf7-9f71-dd53fd1e8772 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:23.849325 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9273af77-854f-4cf7-9f71-dd53fd1e8772 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:23.849325 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9273af77-854f-4cf7-9f71-dd53fd1e8772 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:23.849653 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9273af77-854f-4cf7-9f71-dd53fd1e8772 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:23.849653 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9273af77-854f-4cf7-9f71-dd53fd1e8772 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:23.983650 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-9273af77-854f-4cf7-9f71-dd53fd1e8772 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:23.991050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-9273af77-854f-4cf7-9f71-dd53fd1e8772 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.154397 Jul 05 10:11:23.991558 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 106/214] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:23 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:24.004174 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-42036735-e735-4319-b31d-52399edcfc12 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:24.005889 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-42036735-e735-4319-b31d-52399edcfc12 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:24.011907 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-42036735-e735-4319-b31d-52399edcfc12 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:24.012349 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-42036735-e735-4319-b31d-52399edcfc12 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:24.013126 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-42036735-e735-4319-b31d-52399edcfc12 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:24.013557 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-42036735-e735-4319-b31d-52399edcfc12 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:24.077663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-42036735-e735-4319-b31d-52399edcfc12 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:24.084853 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-42036735-e735-4319-b31d-52399edcfc12 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.084388 Jul 05 10:11:24.085410 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 109/215] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:23 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:24.196234 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0f2e4fb8-714c-4b8f-8e9b-624dcc062d7c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:24.196866 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0f2e4fb8-714c-4b8f-8e9b-624dcc062d7c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:24.203124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0f2e4fb8-714c-4b8f-8e9b-624dcc062d7c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:24.203598 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0f2e4fb8-714c-4b8f-8e9b-624dcc062d7c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:24.204357 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0f2e4fb8-714c-4b8f-8e9b-624dcc062d7c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:24.204819 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0f2e4fb8-714c-4b8f-8e9b-624dcc062d7c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:24.330653 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-0f2e4fb8-714c-4b8f-8e9b-624dcc062d7c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:24.338448 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0f2e4fb8-714c-4b8f-8e9b-624dcc062d7c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.145995 Jul 05 10:11:24.339011 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 107/216] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:24 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:25.007859 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-38d04da2-4cee-4b1f-b2ed-411119ed1081 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:25.008518 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-38d04da2-4cee-4b1f-b2ed-411119ed1081 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:25.014503 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-38d04da2-4cee-4b1f-b2ed-411119ed1081 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:25.014946 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-38d04da2-4cee-4b1f-b2ed-411119ed1081 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:25.015701 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-38d04da2-4cee-4b1f-b2ed-411119ed1081 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:25.016131 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-38d04da2-4cee-4b1f-b2ed-411119ed1081 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:25.101306 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b5a4f0ba-3d8b-4f6d-88d1-6ec0605d4fdf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:25.101904 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b5a4f0ba-3d8b-4f6d-88d1-6ec0605d4fdf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:25.108467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b5a4f0ba-3d8b-4f6d-88d1-6ec0605d4fdf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:25.108912 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b5a4f0ba-3d8b-4f6d-88d1-6ec0605d4fdf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:25.109663 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b5a4f0ba-3d8b-4f6d-88d1-6ec0605d4fdf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:25.110098 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b5a4f0ba-3d8b-4f6d-88d1-6ec0605d4fdf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:25.141707 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-38d04da2-4cee-4b1f-b2ed-411119ed1081 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:25.149066 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-38d04da2-4cee-4b1f-b2ed-411119ed1081 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.144887 Jul 05 10:11:25.149637 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 110/217] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:25 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:25.173742 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b5a4f0ba-3d8b-4f6d-88d1-6ec0605d4fdf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:25.181043 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b5a4f0ba-3d8b-4f6d-88d1-6ec0605d4fdf tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.083488 Jul 05 10:11:25.181597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 108/218] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:25 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:25.355341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e5342c59-1702-43ca-a8a5-a64e880f4f02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:25.355966 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e5342c59-1702-43ca-a8a5-a64e880f4f02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:25.361882 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e5342c59-1702-43ca-a8a5-a64e880f4f02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:25.362321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e5342c59-1702-43ca-a8a5-a64e880f4f02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:25.363068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e5342c59-1702-43ca-a8a5-a64e880f4f02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:25.363498 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e5342c59-1702-43ca-a8a5-a64e880f4f02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:25.487550 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e5342c59-1702-43ca-a8a5-a64e880f4f02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:25.494897 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e5342c59-1702-43ca-a8a5-a64e880f4f02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.143960 Jul 05 10:11:25.495462 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 111/219] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:25 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:26.166997 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-43a92b03-e7a4-4451-a651-64218a374103 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:26.167613 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-43a92b03-e7a4-4451-a651-64218a374103 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:26.173674 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-43a92b03-e7a4-4451-a651-64218a374103 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:26.174122 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-43a92b03-e7a4-4451-a651-64218a374103 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:26.174880 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-43a92b03-e7a4-4451-a651-64218a374103 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:26.175317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-43a92b03-e7a4-4451-a651-64218a374103 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:26.197322 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-08ee0037-57f8-4c38-89d4-5da17239fa31 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:26.197925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-08ee0037-57f8-4c38-89d4-5da17239fa31 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:26.204086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-08ee0037-57f8-4c38-89d4-5da17239fa31 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:26.204569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-08ee0037-57f8-4c38-89d4-5da17239fa31 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:26.205328 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-08ee0037-57f8-4c38-89d4-5da17239fa31 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:26.205766 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-08ee0037-57f8-4c38-89d4-5da17239fa31 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:26.275284 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-08ee0037-57f8-4c38-89d4-5da17239fa31 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:26.282641 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-08ee0037-57f8-4c38-89d4-5da17239fa31 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.089073 Jul 05 10:11:26.283210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 112/220] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:26 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:26.329862 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-43a92b03-e7a4-4451-a651-64218a374103 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:26.337327 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-43a92b03-e7a4-4451-a651-64218a374103 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.174684 Jul 05 10:11:26.337935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 109/221] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:26 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:26.511445 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9ae0c2d8-3564-4c6b-96f4-63252bcbe9e7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:26.512071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9ae0c2d8-3564-4c6b-96f4-63252bcbe9e7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:26.518021 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9ae0c2d8-3564-4c6b-96f4-63252bcbe9e7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:26.518470 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9ae0c2d8-3564-4c6b-96f4-63252bcbe9e7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:26.519224 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9ae0c2d8-3564-4c6b-96f4-63252bcbe9e7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:26.519656 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9ae0c2d8-3564-4c6b-96f4-63252bcbe9e7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:26.644097 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9ae0c2d8-3564-4c6b-96f4-63252bcbe9e7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:26.652076 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9ae0c2d8-3564-4c6b-96f4-63252bcbe9e7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.144439 Jul 05 10:11:26.652655 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 113/222] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:26 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:27.298846 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c21337a4-098e-43e5-93bc-9599b953229b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:27.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-c21337a4-098e-43e5-93bc-9599b953229b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:27.305562 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c21337a4-098e-43e5-93bc-9599b953229b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:27.306015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c21337a4-098e-43e5-93bc-9599b953229b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:27.307271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c21337a4-098e-43e5-93bc-9599b953229b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:27.307703 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c21337a4-098e-43e5-93bc-9599b953229b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:27.353974 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-467281aa-708e-47e5-8fab-33f527a44b87 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:27.354574 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-467281aa-708e-47e5-8fab-33f527a44b87 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:27.360503 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-467281aa-708e-47e5-8fab-33f527a44b87 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:27.360954 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-467281aa-708e-47e5-8fab-33f527a44b87 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:27.361703 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-467281aa-708e-47e5-8fab-33f527a44b87 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:27.362137 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-467281aa-708e-47e5-8fab-33f527a44b87 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:27.367805 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-c21337a4-098e-43e5-93bc-9599b953229b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:27.375705 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-c21337a4-098e-43e5-93bc-9599b953229b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.080756 Jul 05 10:11:27.376260 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 110/223] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:27 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:27.491228 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-467281aa-708e-47e5-8fab-33f527a44b87 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:27.499064 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-467281aa-708e-47e5-8fab-33f527a44b87 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.148919 Jul 05 10:11:27.499620 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 114/224] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:27 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:27.669754 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-9aa30cde-a323-4a16-9a35-b3e66ddb487f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:27.670369 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-9aa30cde-a323-4a16-9a35-b3e66ddb487f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:27.676219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9aa30cde-a323-4a16-9a35-b3e66ddb487f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:27.676691 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9aa30cde-a323-4a16-9a35-b3e66ddb487f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:27.677443 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9aa30cde-a323-4a16-9a35-b3e66ddb487f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:27.677875 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9aa30cde-a323-4a16-9a35-b3e66ddb487f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:27.808330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-9aa30cde-a323-4a16-9a35-b3e66ddb487f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:27.815736 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-9aa30cde-a323-4a16-9a35-b3e66ddb487f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.151142 Jul 05 10:11:27.816249 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 111/225] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:27 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:28.393032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a24d607a-f62c-4509-aa3c-e72f14593ea5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:28.393560 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a24d607a-f62c-4509-aa3c-e72f14593ea5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:28.399533 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a24d607a-f62c-4509-aa3c-e72f14593ea5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:28.399978 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a24d607a-f62c-4509-aa3c-e72f14593ea5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:28.401360 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a24d607a-f62c-4509-aa3c-e72f14593ea5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:28.401797 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a24d607a-f62c-4509-aa3c-e72f14593ea5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:28.461859 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a24d607a-f62c-4509-aa3c-e72f14593ea5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:28.469914 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a24d607a-f62c-4509-aa3c-e72f14593ea5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.080770 Jul 05 10:11:28.470480 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 115/226] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:28 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:28.515388 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fbe46630-b624-4ff2-bd65-b53d46a7833b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:28.515991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fbe46630-b624-4ff2-bd65-b53d46a7833b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:28.522325 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fbe46630-b624-4ff2-bd65-b53d46a7833b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:28.522759 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fbe46630-b624-4ff2-bd65-b53d46a7833b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:28.523525 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fbe46630-b624-4ff2-bd65-b53d46a7833b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:28.523962 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fbe46630-b624-4ff2-bd65-b53d46a7833b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:28.655167 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fbe46630-b624-4ff2-bd65-b53d46a7833b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:28.663059 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fbe46630-b624-4ff2-bd65-b53d46a7833b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.151398 Jul 05 10:11:28.663738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 112/227] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:28 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:28.832436 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-038cb666-6b6e-42f3-ab00-e1e15a1b7022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:28.833078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-038cb666-6b6e-42f3-ab00-e1e15a1b7022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:28.840272 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-038cb666-6b6e-42f3-ab00-e1e15a1b7022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:28.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-038cb666-6b6e-42f3-ab00-e1e15a1b7022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:28.841646 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-038cb666-6b6e-42f3-ab00-e1e15a1b7022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:28.842094 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-038cb666-6b6e-42f3-ab00-e1e15a1b7022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:29.074867 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-4070789a-2009-4735-a091-936408a8b28d req-3c4b1c0f-ba11-4f4f-acbe-222ac9eb0668 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ab80efc4-b2fa-4761-806b-2a197cc5b00b", "name": "network-vif-plugged", "status": "completed", "tag": "66542ef5-f0db-4a7d-ab6e-ceb382db45de"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:11:29.084092 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-4070789a-2009-4735-a091-936408a8b28d req-3c4b1c0f-ba11-4f4f-acbe-222ac9eb0668 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:29.084748 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-4070789a-2009-4735-a091-936408a8b28d req-3c4b1c0f-ba11-4f4f-acbe-222ac9eb0668 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:29.084923 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-038cb666-6b6e-42f3-ab00-e1e15a1b7022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:29.088623 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=100 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:29.094776 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-038cb666-6b6e-42f3-ab00-e1e15a1b7022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.266166 Jul 05 10:11:29.095356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 116/228] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:28 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:29.097479 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-4070789a-2009-4735-a091-936408a8b28d req-3c4b1c0f-ba11-4f4f-acbe-222ac9eb0668 service nova] Creating event network-vif-plugged:66542ef5-f0db-4a7d-ab6e-ceb382db45de for instance ab80efc4-b2fa-4761-806b-2a197cc5b00b on hv-cinder-maste Jul 05 10:11:29.105439 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-4070789a-2009-4735-a091-936408a8b28d req-3c4b1c0f-ba11-4f4f-acbe-222ac9eb0668 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035104 Jul 05 10:11:29.106036 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 113/229] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:11:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:11:29.487633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b9ed260a-46a7-46e8-b5f8-266a836c3559 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:29.489212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b9ed260a-46a7-46e8-b5f8-266a836c3559 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:29.499204 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b9ed260a-46a7-46e8-b5f8-266a836c3559 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:29.499686 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b9ed260a-46a7-46e8-b5f8-266a836c3559 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:29.500454 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b9ed260a-46a7-46e8-b5f8-266a836c3559 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:29.500912 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b9ed260a-46a7-46e8-b5f8-266a836c3559 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:29.561582 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b9ed260a-46a7-46e8-b5f8-266a836c3559 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:29.568782 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b9ed260a-46a7-46e8-b5f8-266a836c3559 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.086081 Jul 05 10:11:29.569341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 117/230] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:29 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:29.680357 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7737fb53-44e7-4d80-bcfc-8659fd42816e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:29.681081 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7737fb53-44e7-4d80-bcfc-8659fd42816e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:29.688046 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7737fb53-44e7-4d80-bcfc-8659fd42816e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:29.688536 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7737fb53-44e7-4d80-bcfc-8659fd42816e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:29.689393 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7737fb53-44e7-4d80-bcfc-8659fd42816e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:29.689825 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7737fb53-44e7-4d80-bcfc-8659fd42816e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:29.707759 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=100 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:29.826681 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-7737fb53-44e7-4d80-bcfc-8659fd42816e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:29.834527 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7737fb53-44e7-4d80-bcfc-8659fd42816e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.158444 Jul 05 10:11:29.844283 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 114/231] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:29 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:30.110623 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a1be99dc-6323-409a-aad2-268be5a9d99e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:30.111317 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a1be99dc-6323-409a-aad2-268be5a9d99e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:30.117377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a1be99dc-6323-409a-aad2-268be5a9d99e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:30.117707 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a1be99dc-6323-409a-aad2-268be5a9d99e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:30.118428 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a1be99dc-6323-409a-aad2-268be5a9d99e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:30.118859 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a1be99dc-6323-409a-aad2-268be5a9d99e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:30.206388 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fc5239a4-d6e8-41c4-998c-6bde446935c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f14c4815-f6cd-4e9d-af4e-27c2f6f39995"}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:11:30.207471 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fc5239a4-d6e8-41c4-998c-6bde446935c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:30.222364 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fc5239a4-d6e8-41c4-998c-6bde446935c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:30.222825 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fc5239a4-d6e8-41c4-998c-6bde446935c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:30.223600 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fc5239a4-d6e8-41c4-998c-6bde446935c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:30.224040 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fc5239a4-d6e8-41c4-998c-6bde446935c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:30.270334 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a1be99dc-6323-409a-aad2-268be5a9d99e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:30.279292 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a1be99dc-6323-409a-aad2-268be5a9d99e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.172350 Jul 05 10:11:30.280001 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 118/232] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:30 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:30.584662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-67d8b9af-13dc-48f1-8615-c07a2ccac512 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:30.585245 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-67d8b9af-13dc-48f1-8615-c07a2ccac512 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:30.591127 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-67d8b9af-13dc-48f1-8615-c07a2ccac512 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:30.591574 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-67d8b9af-13dc-48f1-8615-c07a2ccac512 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:30.592330 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-67d8b9af-13dc-48f1-8615-c07a2ccac512 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:30.592788 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-67d8b9af-13dc-48f1-8615-c07a2ccac512 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:30.652717 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-67d8b9af-13dc-48f1-8615-c07a2ccac512 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:30.659959 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-67d8b9af-13dc-48f1-8615-c07a2ccac512 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.079166 Jul 05 10:11:30.660549 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 119/233] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:30 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:30.851717 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-66ca6a9d-d59a-429d-987e-3a4c9fbeccb2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:30.852255 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-66ca6a9d-d59a-429d-987e-3a4c9fbeccb2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:30.859858 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-66ca6a9d-d59a-429d-987e-3a4c9fbeccb2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-66ca6a9d-d59a-429d-987e-3a4c9fbeccb2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:30.861597 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-66ca6a9d-d59a-429d-987e-3a4c9fbeccb2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:30.862223 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-66ca6a9d-d59a-429d-987e-3a4c9fbeccb2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:30.991456 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-66ca6a9d-d59a-429d-987e-3a4c9fbeccb2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:30.999449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-66ca6a9d-d59a-429d-987e-3a4c9fbeccb2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.151545 Jul 05 10:11:31.000022 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 120/234] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:30 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:31.295999 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-eefc43ac-1b8a-469e-94e7-df2339d446bf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:31.296629 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-eefc43ac-1b8a-469e-94e7-df2339d446bf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:31.302630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-eefc43ac-1b8a-469e-94e7-df2339d446bf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:31.303068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-eefc43ac-1b8a-469e-94e7-df2339d446bf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:31.303813 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-eefc43ac-1b8a-469e-94e7-df2339d446bf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:31.304241 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-eefc43ac-1b8a-469e-94e7-df2339d446bf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:31.433553 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-eefc43ac-1b8a-469e-94e7-df2339d446bf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:31.440797 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-eefc43ac-1b8a-469e-94e7-df2339d446bf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.148565 Jul 05 10:11:31.441373 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 121/235] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:31 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:31.677063 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6f11d057-7257-4ed0-88f6-02aff7ac859c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:31.677627 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-6f11d057-7257-4ed0-88f6-02aff7ac859c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:31.684178 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6f11d057-7257-4ed0-88f6-02aff7ac859c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:31.684647 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6f11d057-7257-4ed0-88f6-02aff7ac859c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:31.685402 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6f11d057-7257-4ed0-88f6-02aff7ac859c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:31.685831 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6f11d057-7257-4ed0-88f6-02aff7ac859c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:31.746010 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6f11d057-7257-4ed0-88f6-02aff7ac859c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:31.753290 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-6f11d057-7257-4ed0-88f6-02aff7ac859c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.080081 Jul 05 10:11:31.753882 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 122/236] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:31 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:32.016067 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-989e4c33-989a-468f-8622-4df948f6d463 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:32.016733 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-989e4c33-989a-468f-8622-4df948f6d463 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:32.022743 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-989e4c33-989a-468f-8622-4df948f6d463 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:32.023188 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-989e4c33-989a-468f-8622-4df948f6d463 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:32.023944 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-989e4c33-989a-468f-8622-4df948f6d463 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:32.024384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-989e4c33-989a-468f-8622-4df948f6d463 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:32.154428 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-989e4c33-989a-468f-8622-4df948f6d463 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:32.162127 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-989e4c33-989a-468f-8622-4df948f6d463 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.149854 Jul 05 10:11:32.162701 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 123/237] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:32 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:32.457483 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-18daa97b-7ed5-4e86-ae6a-b6192114b515 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:32.458121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-18daa97b-7ed5-4e86-ae6a-b6192114b515 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:32.464776 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18daa97b-7ed5-4e86-ae6a-b6192114b515 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:32.465225 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18daa97b-7ed5-4e86-ae6a-b6192114b515 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:32.465982 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18daa97b-7ed5-4e86-ae6a-b6192114b515 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:32.466417 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18daa97b-7ed5-4e86-ae6a-b6192114b515 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:32.591896 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-18daa97b-7ed5-4e86-ae6a-b6192114b515 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:32.599230 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-18daa97b-7ed5-4e86-ae6a-b6192114b515 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.145714 Jul 05 10:11:32.599781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 124/238] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:32 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:32.769378 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-189c7d78-8cc0-4b94-af33-fcf90d118f3e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:32.769978 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-189c7d78-8cc0-4b94-af33-fcf90d118f3e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:32.776185 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-189c7d78-8cc0-4b94-af33-fcf90d118f3e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:32.776679 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-189c7d78-8cc0-4b94-af33-fcf90d118f3e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:32.777463 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-189c7d78-8cc0-4b94-af33-fcf90d118f3e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:32.777894 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-189c7d78-8cc0-4b94-af33-fcf90d118f3e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:32.838913 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-189c7d78-8cc0-4b94-af33-fcf90d118f3e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:32.846396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-189c7d78-8cc0-4b94-af33-fcf90d118f3e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.080787 Jul 05 10:11:32.847033 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 125/239] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:32 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:33.178344 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-800ecab1-91a9-4216-8dbe-92596b2582dc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:33.178945 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-800ecab1-91a9-4216-8dbe-92596b2582dc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:33.185459 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-800ecab1-91a9-4216-8dbe-92596b2582dc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:33.185901 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-800ecab1-91a9-4216-8dbe-92596b2582dc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:33.186654 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-800ecab1-91a9-4216-8dbe-92596b2582dc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:33.187087 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-800ecab1-91a9-4216-8dbe-92596b2582dc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:33.316673 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-800ecab1-91a9-4216-8dbe-92596b2582dc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:33.323914 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-800ecab1-91a9-4216-8dbe-92596b2582dc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.149320 Jul 05 10:11:33.324503 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 126/240] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:33 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:33.614666 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c17fe5b9-fcbd-495a-8ad1-bfc31542f4c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:33.615268 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c17fe5b9-fcbd-495a-8ad1-bfc31542f4c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:33.621188 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c17fe5b9-fcbd-495a-8ad1-bfc31542f4c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:33.621636 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c17fe5b9-fcbd-495a-8ad1-bfc31542f4c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:33.622386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c17fe5b9-fcbd-495a-8ad1-bfc31542f4c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:33.622823 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c17fe5b9-fcbd-495a-8ad1-bfc31542f4c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:33.746479 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c17fe5b9-fcbd-495a-8ad1-bfc31542f4c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:33.753752 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c17fe5b9-fcbd-495a-8ad1-bfc31542f4c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.142807 Jul 05 10:11:33.754313 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 127/241] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:33 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:33.863170 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-97079f7d-f6e5-4c28-abf3-a89e564f137b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:33.863766 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-97079f7d-f6e5-4c28-abf3-a89e564f137b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:33.869553 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-97079f7d-f6e5-4c28-abf3-a89e564f137b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:33.870023 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-97079f7d-f6e5-4c28-abf3-a89e564f137b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:33.870776 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-97079f7d-f6e5-4c28-abf3-a89e564f137b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:33.871208 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-97079f7d-f6e5-4c28-abf3-a89e564f137b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:33.931842 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-97079f7d-f6e5-4c28-abf3-a89e564f137b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:33.939428 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-97079f7d-f6e5-4c28-abf3-a89e564f137b tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.079980 Jul 05 10:11:33.939990 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 128/242] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:33 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:34.340829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f0cf0196-7a33-45d2-9116-bc5048ab01c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:34.341433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f0cf0196-7a33-45d2-9116-bc5048ab01c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:34.347316 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f0cf0196-7a33-45d2-9116-bc5048ab01c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:34.347760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f0cf0196-7a33-45d2-9116-bc5048ab01c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:34.348532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f0cf0196-7a33-45d2-9116-bc5048ab01c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:34.348962 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f0cf0196-7a33-45d2-9116-bc5048ab01c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:34.477604 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f0cf0196-7a33-45d2-9116-bc5048ab01c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:34.485489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f0cf0196-7a33-45d2-9116-bc5048ab01c2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.148619 Jul 05 10:11:34.486049 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 129/243] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:34 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:34.770281 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-79c944ed-57ac-41a4-b0a7-b01366d485d7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:34.770884 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-79c944ed-57ac-41a4-b0a7-b01366d485d7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:34.777361 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-79c944ed-57ac-41a4-b0a7-b01366d485d7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:34.777834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-79c944ed-57ac-41a4-b0a7-b01366d485d7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:34.778584 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-79c944ed-57ac-41a4-b0a7-b01366d485d7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:34.779011 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-79c944ed-57ac-41a4-b0a7-b01366d485d7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:34.969001 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-79c944ed-57ac-41a4-b0a7-b01366d485d7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:34.976270 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-79c944ed-57ac-41a4-b0a7-b01366d485d7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1322 microversion: 2.1 time: 0.209792 Jul 05 10:11:34.986349 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 130/244] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:34 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1322 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:34.986349 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5dbdb557-7889-4a59-b994-48f165a481cd tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:34.986349 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5dbdb557-7889-4a59-b994-48f165a481cd tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:34.990539 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5dbdb557-7889-4a59-b994-48f165a481cd tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:34.990994 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5dbdb557-7889-4a59-b994-48f165a481cd tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:34.991735 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5dbdb557-7889-4a59-b994-48f165a481cd tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:34.992162 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5dbdb557-7889-4a59-b994-48f165a481cd tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:35.081691 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5dbdb557-7889-4a59-b994-48f165a481cd tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:35.089536 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5dbdb557-7889-4a59-b994-48f165a481cd tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1274 microversion: 2.1 time: 0.111232 Jul 05 10:11:35.090098 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 131/245] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:34 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1274 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:35.501657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c7c96742-836b-4bc8-a74b-6eb67e53b5ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:35.502258 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c7c96742-836b-4bc8-a74b-6eb67e53b5ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:35.508126 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c7c96742-836b-4bc8-a74b-6eb67e53b5ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:35.508605 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c7c96742-836b-4bc8-a74b-6eb67e53b5ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:35.509354 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c7c96742-836b-4bc8-a74b-6eb67e53b5ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:35.509811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c7c96742-836b-4bc8-a74b-6eb67e53b5ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:35.642985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c7c96742-836b-4bc8-a74b-6eb67e53b5ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:35.650802 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c7c96742-836b-4bc8-a74b-6eb67e53b5ad tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.152954 Jul 05 10:11:35.659958 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 132/246] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:35 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:35.993995 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-930d7967-da3e-467a-b2ff-16063e177d35 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:35.994620 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-930d7967-da3e-467a-b2ff-16063e177d35 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:36.001305 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-930d7967-da3e-467a-b2ff-16063e177d35 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:36.001744 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-930d7967-da3e-467a-b2ff-16063e177d35 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:36.002499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-930d7967-da3e-467a-b2ff-16063e177d35 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:36.002931 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-930d7967-da3e-467a-b2ff-16063e177d35 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:36.024039 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=100 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:36.133733 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-930d7967-da3e-467a-b2ff-16063e177d35 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:36.141196 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-930d7967-da3e-467a-b2ff-16063e177d35 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1516 microversion: 2.1 time: 0.151073 Jul 05 10:11:36.141763 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 133/247] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:35 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1516 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:36.146377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d85d5d04-2ec9-4db3-a712-e09f10222371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:36.146974 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d85d5d04-2ec9-4db3-a712-e09f10222371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:36.152932 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d85d5d04-2ec9-4db3-a712-e09f10222371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:36.153379 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d85d5d04-2ec9-4db3-a712-e09f10222371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:36.154146 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d85d5d04-2ec9-4db3-a712-e09f10222371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:36.154581 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d85d5d04-2ec9-4db3-a712-e09f10222371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:36.285145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d85d5d04-2ec9-4db3-a712-e09f10222371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:36.292536 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d85d5d04-2ec9-4db3-a712-e09f10222371 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1316 microversion: 2.1 time: 0.149796 Jul 05 10:11:36.293114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 134/248] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:36 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1316 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:36.297903 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2f7f4936-9490-4854-8f94-1f101afbe6df tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:11:36.298918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2f7f4936-9490-4854-8f94-1f101afbe6df tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:36.304809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f7f4936-9490-4854-8f94-1f101afbe6df tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:36.305259 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f7f4936-9490-4854-8f94-1f101afbe6df tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:36.306014 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f7f4936-9490-4854-8f94-1f101afbe6df tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:36.306454 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f7f4936-9490-4854-8f94-1f101afbe6df tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:36.367078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.objects.instance [None req-2f7f4936-9490-4854-8f94-1f101afbe6df tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lazy-loading 'flavor' on Instance uuid fc065612-7f7d-45fb-89cc-9f332fb15dd1 {{(pid=117813) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:11:36.482120 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2f7f4936-9490-4854-8f94-1f101afbe6df tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "POST /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1/action" status: 200 len: 32 microversion: 2.1 time: 0.187772 Jul 05 10:11:36.482836 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 135/249] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:11:36 2022] POST /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1/action => generated 32 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 05 10:11:36.495949 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f7ae8d2d-492e-4886-9ce0-a30edd49a4f9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:36.496710 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f7ae8d2d-492e-4886-9ce0-a30edd49a4f9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:36.502802 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f7ae8d2d-492e-4886-9ce0-a30edd49a4f9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:36.503251 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f7ae8d2d-492e-4886-9ce0-a30edd49a4f9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:36.504004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f7ae8d2d-492e-4886-9ce0-a30edd49a4f9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:36.504465 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f7ae8d2d-492e-4886-9ce0-a30edd49a4f9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:36.636243 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f7ae8d2d-492e-4886-9ce0-a30edd49a4f9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:36.643625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f7ae8d2d-492e-4886-9ce0-a30edd49a4f9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.155420 Jul 05 10:11:36.644190 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 136/250] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:36 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:36.666255 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a5a8cb9a-72ea-4e96-8510-142902fec897 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:36.667339 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a5a8cb9a-72ea-4e96-8510-142902fec897 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:36.673043 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a5a8cb9a-72ea-4e96-8510-142902fec897 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:36.673483 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a5a8cb9a-72ea-4e96-8510-142902fec897 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:36.674231 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a5a8cb9a-72ea-4e96-8510-142902fec897 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:36.674659 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a5a8cb9a-72ea-4e96-8510-142902fec897 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:36.806496 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a5a8cb9a-72ea-4e96-8510-142902fec897 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:36.814502 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a5a8cb9a-72ea-4e96-8510-142902fec897 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.151661 Jul 05 10:11:36.815067 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 137/251] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:36 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:37.308779 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-45259e5e-006d-4297-8409-d8e19c08efce tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:37.309385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-45259e5e-006d-4297-8409-d8e19c08efce tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:37.315361 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-45259e5e-006d-4297-8409-d8e19c08efce tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:37.315807 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-45259e5e-006d-4297-8409-d8e19c08efce tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:37.316588 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-45259e5e-006d-4297-8409-d8e19c08efce tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:37.317023 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-45259e5e-006d-4297-8409-d8e19c08efce tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:37.597127 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-45259e5e-006d-4297-8409-d8e19c08efce tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:37.604508 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-45259e5e-006d-4297-8409-d8e19c08efce tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1316 microversion: 2.1 time: 0.299491 Jul 05 10:11:37.605087 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 138/252] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:37 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1316 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:37.659400 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c91d5abd-64bc-4e7d-bc55-927f7923006f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:37.660021 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c91d5abd-64bc-4e7d-bc55-927f7923006f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:37.665964 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c91d5abd-64bc-4e7d-bc55-927f7923006f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:37.666413 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c91d5abd-64bc-4e7d-bc55-927f7923006f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:37.667165 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c91d5abd-64bc-4e7d-bc55-927f7923006f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:37.667600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c91d5abd-64bc-4e7d-bc55-927f7923006f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:37.791403 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c91d5abd-64bc-4e7d-bc55-927f7923006f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:37.798707 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c91d5abd-64bc-4e7d-bc55-927f7923006f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.143059 Jul 05 10:11:37.799267 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 139/253] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:37 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:37.831612 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ae58491e-92b4-4925-93b9-b2249c87b877 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:37.832216 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ae58491e-92b4-4925-93b9-b2249c87b877 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:37.838145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae58491e-92b4-4925-93b9-b2249c87b877 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:37.838591 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae58491e-92b4-4925-93b9-b2249c87b877 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:37.839354 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae58491e-92b4-4925-93b9-b2249c87b877 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:37.839792 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae58491e-92b4-4925-93b9-b2249c87b877 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:37.969139 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ae58491e-92b4-4925-93b9-b2249c87b877 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:37.976367 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ae58491e-92b4-4925-93b9-b2249c87b877 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.148983 Jul 05 10:11:37.976947 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 140/254] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:37 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:38.621009 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5f7ef7eb-9c19-49ee-8895-c9b7348687db tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:38.621604 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5f7ef7eb-9c19-49ee-8895-c9b7348687db tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:38.627585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5f7ef7eb-9c19-49ee-8895-c9b7348687db tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:38.628034 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5f7ef7eb-9c19-49ee-8895-c9b7348687db tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:38.628826 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5f7ef7eb-9c19-49ee-8895-c9b7348687db tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:38.629328 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5f7ef7eb-9c19-49ee-8895-c9b7348687db tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:38.752649 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5f7ef7eb-9c19-49ee-8895-c9b7348687db tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:38.759972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5f7ef7eb-9c19-49ee-8895-c9b7348687db tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1316 microversion: 2.1 time: 0.142753 Jul 05 10:11:38.760564 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 141/255] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:38 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1316 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:38.815536 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0775858d-63e1-4da9-8f65-55fdcd1f21f3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:38.816164 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0775858d-63e1-4da9-8f65-55fdcd1f21f3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:38.818007 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=100 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:38.823282 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0775858d-63e1-4da9-8f65-55fdcd1f21f3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:38.823772 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0775858d-63e1-4da9-8f65-55fdcd1f21f3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:38.824505 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0775858d-63e1-4da9-8f65-55fdcd1f21f3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:38.824942 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0775858d-63e1-4da9-8f65-55fdcd1f21f3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:38.955426 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0775858d-63e1-4da9-8f65-55fdcd1f21f3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:38.962877 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0775858d-63e1-4da9-8f65-55fdcd1f21f3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.150865 Jul 05 10:11:38.963434 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 142/256] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:38 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:38.992373 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-17121472-9190-4cb9-8f11-4998ba638b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:38.992988 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-17121472-9190-4cb9-8f11-4998ba638b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:38.998596 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-17121472-9190-4cb9-8f11-4998ba638b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:38.999042 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-17121472-9190-4cb9-8f11-4998ba638b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:38.999794 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-17121472-9190-4cb9-8f11-4998ba638b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:39.000227 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-17121472-9190-4cb9-8f11-4998ba638b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:39.127519 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-17121472-9190-4cb9-8f11-4998ba638b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:39.134831 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-17121472-9190-4cb9-8f11-4998ba638b3e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.145867 Jul 05 10:11:39.135395 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 143/257] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:38 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:39.775819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d0e7b239-208a-4cfe-b5a6-4fd4967e7df8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:39.776439 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d0e7b239-208a-4cfe-b5a6-4fd4967e7df8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:39.782359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d0e7b239-208a-4cfe-b5a6-4fd4967e7df8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:39.782808 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d0e7b239-208a-4cfe-b5a6-4fd4967e7df8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:39.783565 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d0e7b239-208a-4cfe-b5a6-4fd4967e7df8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:39.783999 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d0e7b239-208a-4cfe-b5a6-4fd4967e7df8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:39.909063 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d0e7b239-208a-4cfe-b5a6-4fd4967e7df8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:39.917626 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d0e7b239-208a-4cfe-b5a6-4fd4967e7df8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1316 microversion: 2.1 time: 0.145052 Jul 05 10:11:39.918207 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 144/258] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:39 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1316 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:39.978956 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f28be2ae-f8f2-4773-a1ed-b1498faa39fa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:39.980222 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f28be2ae-f8f2-4773-a1ed-b1498faa39fa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:39.986132 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f28be2ae-f8f2-4773-a1ed-b1498faa39fa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:39.986573 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f28be2ae-f8f2-4773-a1ed-b1498faa39fa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:39.987326 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f28be2ae-f8f2-4773-a1ed-b1498faa39fa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:39.987757 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f28be2ae-f8f2-4773-a1ed-b1498faa39fa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:40.117562 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f28be2ae-f8f2-4773-a1ed-b1498faa39fa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:40.125122 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f28be2ae-f8f2-4773-a1ed-b1498faa39fa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.149906 Jul 05 10:11:40.125734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 145/259] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:39 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:40.159032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-fe711e63-abe6-488b-a879-91920cc2744d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:40.159652 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-fe711e63-abe6-488b-a879-91920cc2744d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:40.169174 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fe711e63-abe6-488b-a879-91920cc2744d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:40.169514 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fe711e63-abe6-488b-a879-91920cc2744d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:40.170345 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fe711e63-abe6-488b-a879-91920cc2744d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:40.170788 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fe711e63-abe6-488b-a879-91920cc2744d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:40.211958 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=51 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:40.341935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=8 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:40.394452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-fe711e63-abe6-488b-a879-91920cc2744d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:40.403473 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-fe711e63-abe6-488b-a879-91920cc2744d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.248295 Jul 05 10:11:40.404113 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 146/260] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:40 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:40.778370 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-d38e9f22-18a7-4383-8f35-d0b5a1751be4 req-6c3505b0-5c64-43a3-be2e-66178383e068 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbea07d3-b6f1-479e-9fcb-abcfa716bb5b", "tag": "4e423f85-def7-4513-9e8f-09149af663a9"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:11:40.786388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-d38e9f22-18a7-4383-8f35-d0b5a1751be4 req-6c3505b0-5c64-43a3-be2e-66178383e068 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:40.786914 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-d38e9f22-18a7-4383-8f35-d0b5a1751be4 req-6c3505b0-5c64-43a3-be2e-66178383e068 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:40.798770 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-d38e9f22-18a7-4383-8f35-d0b5a1751be4 req-6c3505b0-5c64-43a3-be2e-66178383e068 service nova] Creating event network-changed:4e423f85-def7-4513-9e8f-09149af663a9 for instance fbea07d3-b6f1-479e-9fcb-abcfa716bb5b on hv-cinder-maste Jul 05 10:11:40.803636 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-d38e9f22-18a7-4383-8f35-d0b5a1751be4 req-6c3505b0-5c64-43a3-be2e-66178383e068 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.028969 Jul 05 10:11:40.804298 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 147/261] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:11:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:11:40.934818 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d8e5edb7-5613-43c2-9682-c1a4e5f8bd9a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:40.935503 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d8e5edb7-5613-43c2-9682-c1a4e5f8bd9a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:40.941934 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8e5edb7-5613-43c2-9682-c1a4e5f8bd9a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:40.942375 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8e5edb7-5613-43c2-9682-c1a4e5f8bd9a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:40.943126 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8e5edb7-5613-43c2-9682-c1a4e5f8bd9a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:40.943554 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8e5edb7-5613-43c2-9682-c1a4e5f8bd9a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:41.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d8e5edb7-5613-43c2-9682-c1a4e5f8bd9a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:41.098159 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:41.103353 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d8e5edb7-5613-43c2-9682-c1a4e5f8bd9a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1326 microversion: 2.1 time: 0.173113 Jul 05 10:11:41.104043 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 148/262] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:40 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1326 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:41.145894 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-93a2374d-7097-4091-b4a5-e57a8fb26f85 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:41.146659 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-93a2374d-7097-4091-b4a5-e57a8fb26f85 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:41.157833 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-93a2374d-7097-4091-b4a5-e57a8fb26f85 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:41.158281 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-93a2374d-7097-4091-b4a5-e57a8fb26f85 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:41.159139 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-93a2374d-7097-4091-b4a5-e57a8fb26f85 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:41.159686 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-93a2374d-7097-4091-b4a5-e57a8fb26f85 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:41.328178 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-93a2374d-7097-4091-b4a5-e57a8fb26f85 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:41.335790 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-93a2374d-7097-4091-b4a5-e57a8fb26f85 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.197567 Jul 05 10:11:41.336577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 149/263] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:41 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:41.421022 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-677e089a-1aac-48af-85c2-6403897fba4a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:41.421750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-677e089a-1aac-48af-85c2-6403897fba4a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:41.427755 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-677e089a-1aac-48af-85c2-6403897fba4a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:41.428203 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-677e089a-1aac-48af-85c2-6403897fba4a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:41.429000 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-677e089a-1aac-48af-85c2-6403897fba4a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:41.429431 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-677e089a-1aac-48af-85c2-6403897fba4a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:41.594489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-677e089a-1aac-48af-85c2-6403897fba4a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:41.602496 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-677e089a-1aac-48af-85c2-6403897fba4a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.185732 Jul 05 10:11:41.603258 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 150/264] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:41 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:42.122015 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-12aa2fa3-2555-42c9-88f4-fa1f64433000 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:42.122626 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-12aa2fa3-2555-42c9-88f4-fa1f64433000 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:42.130644 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-12aa2fa3-2555-42c9-88f4-fa1f64433000 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:42.137744 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-12aa2fa3-2555-42c9-88f4-fa1f64433000 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:42.138677 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-12aa2fa3-2555-42c9-88f4-fa1f64433000 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:42.139480 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-12aa2fa3-2555-42c9-88f4-fa1f64433000 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:42.307251 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-12aa2fa3-2555-42c9-88f4-fa1f64433000 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:42.316550 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-12aa2fa3-2555-42c9-88f4-fa1f64433000 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.199222 Jul 05 10:11:42.317280 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 151/265] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:42 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:42.352914 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3475c090-e652-433e-bd9b-2cffb579aa65 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:42.353615 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3475c090-e652-433e-bd9b-2cffb579aa65 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:42.360278 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3475c090-e652-433e-bd9b-2cffb579aa65 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:42.360881 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3475c090-e652-433e-bd9b-2cffb579aa65 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:42.361711 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3475c090-e652-433e-bd9b-2cffb579aa65 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:42.362361 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3475c090-e652-433e-bd9b-2cffb579aa65 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:42.501709 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-3475c090-e652-433e-bd9b-2cffb579aa65 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:42.509181 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3475c090-e652-433e-bd9b-2cffb579aa65 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.160324 Jul 05 10:11:42.509825 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 152/266] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:42 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:42.618827 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ccde722d-968d-46e8-a644-c6588e3bd5fa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:42.619423 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ccde722d-968d-46e8-a644-c6588e3bd5fa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:42.626186 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ccde722d-968d-46e8-a644-c6588e3bd5fa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:42.626618 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ccde722d-968d-46e8-a644-c6588e3bd5fa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:42.627375 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ccde722d-968d-46e8-a644-c6588e3bd5fa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:42.627798 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ccde722d-968d-46e8-a644-c6588e3bd5fa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:42.848068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ccde722d-968d-46e8-a644-c6588e3bd5fa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:42.860815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ccde722d-968d-46e8-a644-c6588e3bd5fa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1366 microversion: 2.1 time: 0.242379 Jul 05 10:11:42.860815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 153/267] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:42 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1366 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:43.336129 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a8cdbbf5-f8cd-43c8-8c31-594c2e06cc61 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:43.337459 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a8cdbbf5-f8cd-43c8-8c31-594c2e06cc61 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:43.351319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a8cdbbf5-f8cd-43c8-8c31-594c2e06cc61 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:43.352760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a8cdbbf5-f8cd-43c8-8c31-594c2e06cc61 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:43.355725 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a8cdbbf5-f8cd-43c8-8c31-594c2e06cc61 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:43.357162 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a8cdbbf5-f8cd-43c8-8c31-594c2e06cc61 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:43.606882 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fc5239a4-d6e8-41c4-998c-6bde446935c9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "POST /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 13.404076 Jul 05 10:11:43.608568 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 115/268] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:11:30 2022] POST /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857/os-volume_attachments => generated 194 bytes in 13406 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:11:43.613806 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-22ffefe5-674c-40bf-a46b-a8961f20cad5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:43.614702 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-22ffefe5-674c-40bf-a46b-a8961f20cad5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:43.629246 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22ffefe5-674c-40bf-a46b-a8961f20cad5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:43.629706 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22ffefe5-674c-40bf-a46b-a8961f20cad5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:43.630491 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22ffefe5-674c-40bf-a46b-a8961f20cad5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:43.634603 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22ffefe5-674c-40bf-a46b-a8961f20cad5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:44.018683 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a8cdbbf5-f8cd-43c8-8c31-594c2e06cc61 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:44.033651 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a8cdbbf5-f8cd-43c8-8c31-594c2e06cc61 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.702013 Jul 05 10:11:44.045968 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 154/269] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:43 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 716 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:44.053834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-23b93bb9-5229-4ca4-85f5-441735a1e6a5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:44.056848 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-23b93bb9-5229-4ca4-85f5-441735a1e6a5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:44.067721 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-23b93bb9-5229-4ca4-85f5-441735a1e6a5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:44.068505 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-23b93bb9-5229-4ca4-85f5-441735a1e6a5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:44.069673 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-23b93bb9-5229-4ca4-85f5-441735a1e6a5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:44.070502 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-23b93bb9-5229-4ca4-85f5-441735a1e6a5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:44.076136 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-22ffefe5-674c-40bf-a46b-a8961f20cad5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:44.092049 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-22ffefe5-674c-40bf-a46b-a8961f20cad5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.481498 Jul 05 10:11:44.093524 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 116/270] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:43 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:44.235423 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-23b93bb9-5229-4ca4-85f5-441735a1e6a5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:44.243263 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-23b93bb9-5229-4ca4-85f5-441735a1e6a5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1556 microversion: 2.1 time: 0.194699 Jul 05 10:11:44.243858 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 155/271] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:44 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1556 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:44.261158 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-17b421aa-c4db-4978-af02-99ddece21c86 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:44.261657 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-17b421aa-c4db-4978-af02-99ddece21c86 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:44.267740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-17b421aa-c4db-4978-af02-99ddece21c86 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:44.268301 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-17b421aa-c4db-4978-af02-99ddece21c86 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:44.269184 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-17b421aa-c4db-4978-af02-99ddece21c86 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:44.269689 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-17b421aa-c4db-4978-af02-99ddece21c86 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:44.364854 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-17b421aa-c4db-4978-af02-99ddece21c86 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.107536 Jul 05 10:11:44.365630 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 117/272] 10.222.0.20 () {58 vars in 1274 bytes} [Tue Jul 5 10:11:44 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b/os-interface => generated 285 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:11:45.051968 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-50badf33-4da9-4303-b73f-2812a6f8d5f7 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:45.053469 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-50badf33-4da9-4303-b73f-2812a6f8d5f7 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:45.062246 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-50badf33-4da9-4303-b73f-2812a6f8d5f7 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:45.063678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-50badf33-4da9-4303-b73f-2812a6f8d5f7 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:45.064767 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-50badf33-4da9-4303-b73f-2812a6f8d5f7 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:45.065351 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-50badf33-4da9-4303-b73f-2812a6f8d5f7 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:45.111193 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5fa0b355-bda4-4da2-bcc5-2a5db31087bc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:45.111917 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5fa0b355-bda4-4da2-bcc5-2a5db31087bc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:45.122297 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5fa0b355-bda4-4da2-bcc5-2a5db31087bc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:45.122782 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5fa0b355-bda4-4da2-bcc5-2a5db31087bc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:45.123570 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5fa0b355-bda4-4da2-bcc5-2a5db31087bc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:45.124005 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5fa0b355-bda4-4da2-bcc5-2a5db31087bc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:46.001216 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-50badf33-4da9-4303-b73f-2812a6f8d5f7 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:46.014034 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-50badf33-4da9-4303-b73f-2812a6f8d5f7 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.965872 Jul 05 10:11:46.015023 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 156/273] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:45 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 968 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:46.051588 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5fa0b355-bda4-4da2-bcc5-2a5db31087bc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:46.061498 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5fa0b355-bda4-4da2-bcc5-2a5db31087bc tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.953250 Jul 05 10:11:46.076084 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 118/274] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:45 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 970 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:47.033882 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9c784da9-ddd1-46f7-a89a-ebd625dee679 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:47.033882 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9c784da9-ddd1-46f7-a89a-ebd625dee679 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:47.042301 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9c784da9-ddd1-46f7-a89a-ebd625dee679 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:47.043078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9c784da9-ddd1-46f7-a89a-ebd625dee679 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:47.046037 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9c784da9-ddd1-46f7-a89a-ebd625dee679 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:47.046639 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9c784da9-ddd1-46f7-a89a-ebd625dee679 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:47.086985 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ceb2a6d2-184e-49df-86f7-d4a13dfad49d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:47.090197 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ceb2a6d2-184e-49df-86f7-d4a13dfad49d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:47.110670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ceb2a6d2-184e-49df-86f7-d4a13dfad49d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:47.111800 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ceb2a6d2-184e-49df-86f7-d4a13dfad49d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:47.112736 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ceb2a6d2-184e-49df-86f7-d4a13dfad49d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:47.113448 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ceb2a6d2-184e-49df-86f7-d4a13dfad49d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:47.409109 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9c784da9-ddd1-46f7-a89a-ebd625dee679 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:47.418777 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9c784da9-ddd1-46f7-a89a-ebd625dee679 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.391451 Jul 05 10:11:47.419455 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 157/275] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:47 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:47.438653 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ceb2a6d2-184e-49df-86f7-d4a13dfad49d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:47.449989 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ceb2a6d2-184e-49df-86f7-d4a13dfad49d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.370298 Jul 05 10:11:47.450786 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 119/276] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:47 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:47.628680 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-9c9d4987-1876-4b70-a220-ad14daf97b8e req-9c1fe698-ed5a-46ca-9bfc-9ea7706b4329 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ab80efc4-b2fa-4761-806b-2a197cc5b00b", "tag": "66542ef5-f0db-4a7d-ab6e-ceb382db45de"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:11:47.649465 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-9c9d4987-1876-4b70-a220-ad14daf97b8e req-9c1fe698-ed5a-46ca-9bfc-9ea7706b4329 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:47.649914 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-9c9d4987-1876-4b70-a220-ad14daf97b8e req-9c1fe698-ed5a-46ca-9bfc-9ea7706b4329 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:47.669240 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-9c9d4987-1876-4b70-a220-ad14daf97b8e req-9c1fe698-ed5a-46ca-9bfc-9ea7706b4329 service nova] Creating event network-changed:66542ef5-f0db-4a7d-ab6e-ceb382db45de for instance ab80efc4-b2fa-4761-806b-2a197cc5b00b on hv-cinder-maste Jul 05 10:11:47.682896 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-9c9d4987-1876-4b70-a220-ad14daf97b8e req-9c1fe698-ed5a-46ca-9bfc-9ea7706b4329 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059213 Jul 05 10:11:47.683507 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 158/277] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:11:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:11:48.438894 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2301c3fd-72fa-424e-b617-01c080e09aa4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:48.440111 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2301c3fd-72fa-424e-b617-01c080e09aa4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:48.449091 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2301c3fd-72fa-424e-b617-01c080e09aa4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:48.449852 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2301c3fd-72fa-424e-b617-01c080e09aa4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:48.451187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2301c3fd-72fa-424e-b617-01c080e09aa4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:48.452372 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2301c3fd-72fa-424e-b617-01c080e09aa4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:48.469743 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d68360c6-0c49-4537-a269-53c86445bf67 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:48.473155 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d68360c6-0c49-4537-a269-53c86445bf67 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:48.484622 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d68360c6-0c49-4537-a269-53c86445bf67 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:48.485434 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d68360c6-0c49-4537-a269-53c86445bf67 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:48.487577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d68360c6-0c49-4537-a269-53c86445bf67 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:48.491920 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d68360c6-0c49-4537-a269-53c86445bf67 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:48.730154 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d68360c6-0c49-4537-a269-53c86445bf67 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:48.733870 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-2301c3fd-72fa-424e-b617-01c080e09aa4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:48.744265 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2301c3fd-72fa-424e-b617-01c080e09aa4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.309469 Jul 05 10:11:48.746926 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d68360c6-0c49-4537-a269-53c86445bf67 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.283065 Jul 05 10:11:48.750638 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 159/278] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:48 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:48.753263 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 120/279] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:48 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:49.763526 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-67f5026b-96ea-40a4-814e-7017197aa935 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:49.764137 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-67f5026b-96ea-40a4-814e-7017197aa935 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:49.766807 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d9f40e01-b1ba-424f-af07-7554fb83d698 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:49.767445 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d9f40e01-b1ba-424f-af07-7554fb83d698 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:49.775272 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d9f40e01-b1ba-424f-af07-7554fb83d698 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:49.775721 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d9f40e01-b1ba-424f-af07-7554fb83d698 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:49.779964 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d9f40e01-b1ba-424f-af07-7554fb83d698 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:49.780832 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d9f40e01-b1ba-424f-af07-7554fb83d698 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:49.785576 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-67f5026b-96ea-40a4-814e-7017197aa935 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:49.786048 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-67f5026b-96ea-40a4-814e-7017197aa935 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:49.786845 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-67f5026b-96ea-40a4-814e-7017197aa935 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:49.787286 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-67f5026b-96ea-40a4-814e-7017197aa935 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:50.117581 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d9f40e01-b1ba-424f-af07-7554fb83d698 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:50.131962 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d9f40e01-b1ba-424f-af07-7554fb83d698 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.367438 Jul 05 10:11:50.137174 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-67f5026b-96ea-40a4-814e-7017197aa935 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:50.146345 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 121/280] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:49 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:50.151292 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-67f5026b-96ea-40a4-814e-7017197aa935 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.391511 Jul 05 10:11:50.152940 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 160/281] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:49 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:51.152890 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2c1859ac-df01-471f-b934-6b8e39273c02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:51.153573 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2c1859ac-df01-471f-b934-6b8e39273c02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:51.160182 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2c1859ac-df01-471f-b934-6b8e39273c02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:51.160670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2c1859ac-df01-471f-b934-6b8e39273c02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:51.161445 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2c1859ac-df01-471f-b934-6b8e39273c02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:51.161880 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2c1859ac-df01-471f-b934-6b8e39273c02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:51.168848 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c9c65bcc-ff7b-491c-940e-dacd334d4ae3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:51.169472 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c9c65bcc-ff7b-491c-940e-dacd334d4ae3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:51.179906 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c9c65bcc-ff7b-491c-940e-dacd334d4ae3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:51.179906 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c9c65bcc-ff7b-491c-940e-dacd334d4ae3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:51.192854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c9c65bcc-ff7b-491c-940e-dacd334d4ae3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:51.194594 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c9c65bcc-ff7b-491c-940e-dacd334d4ae3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:51.502600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c9c65bcc-ff7b-491c-940e-dacd334d4ae3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:51.508902 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-2c1859ac-df01-471f-b934-6b8e39273c02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:51.513252 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c9c65bcc-ff7b-491c-940e-dacd334d4ae3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.348941 Jul 05 10:11:51.514474 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 161/282] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:51 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:51.525910 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2c1859ac-df01-471f-b934-6b8e39273c02 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.377693 Jul 05 10:11:51.526478 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 122/283] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:51 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:52.532431 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-72a2ff99-45ff-4cd1-8694-d6f6896c8a10 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:52.533816 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-72a2ff99-45ff-4cd1-8694-d6f6896c8a10 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:52.543097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-389abde3-a28d-40fd-853b-bc06841fca17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:52.543711 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-389abde3-a28d-40fd-853b-bc06841fca17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:52.551133 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-72a2ff99-45ff-4cd1-8694-d6f6896c8a10 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:52.551587 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-72a2ff99-45ff-4cd1-8694-d6f6896c8a10 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:52.552376 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-72a2ff99-45ff-4cd1-8694-d6f6896c8a10 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:52.556238 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-389abde3-a28d-40fd-853b-bc06841fca17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:52.565317 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-72a2ff99-45ff-4cd1-8694-d6f6896c8a10 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:52.565862 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-389abde3-a28d-40fd-853b-bc06841fca17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:52.567087 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-389abde3-a28d-40fd-853b-bc06841fca17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:52.567731 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-389abde3-a28d-40fd-853b-bc06841fca17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:52.942998 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-72a2ff99-45ff-4cd1-8694-d6f6896c8a10 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:52.946033 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-389abde3-a28d-40fd-853b-bc06841fca17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:52.951972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-72a2ff99-45ff-4cd1-8694-d6f6896c8a10 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.425559 Jul 05 10:11:52.952921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 162/284] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:52 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:52.954193 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-389abde3-a28d-40fd-853b-bc06841fca17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.415101 Jul 05 10:11:52.954747 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 123/285] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:52 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:53.537847 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-e3bb8db4-45e3-45e2-b82b-a74c24344c29 req-71452df6-a9c0-4ce3-be03-8b6425360872 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbea07d3-b6f1-479e-9fcb-abcfa716bb5b", "name": "network-vif-plugged", "status": "completed", "tag": "4e423f85-def7-4513-9e8f-09149af663a9"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:11:53.549120 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-e3bb8db4-45e3-45e2-b82b-a74c24344c29 req-71452df6-a9c0-4ce3-be03-8b6425360872 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:53.549766 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-e3bb8db4-45e3-45e2-b82b-a74c24344c29 req-71452df6-a9c0-4ce3-be03-8b6425360872 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:53.569138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-e3bb8db4-45e3-45e2-b82b-a74c24344c29 req-71452df6-a9c0-4ce3-be03-8b6425360872 service nova] Creating event network-vif-plugged:4e423f85-def7-4513-9e8f-09149af663a9 for instance fbea07d3-b6f1-479e-9fcb-abcfa716bb5b on hv-cinder-maste Jul 05 10:11:53.584006 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-e3bb8db4-45e3-45e2-b82b-a74c24344c29 req-71452df6-a9c0-4ce3-be03-8b6425360872 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050946 Jul 05 10:11:53.586094 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 163/286] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:11:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:11:53.970143 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7da5d462-1597-4bd7-8176-eb46e3a02d4f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:53.970746 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7da5d462-1597-4bd7-8176-eb46e3a02d4f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:53.971898 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-35766a78-5512-466f-b26d-bdd4831ae619 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:53.972601 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-35766a78-5512-466f-b26d-bdd4831ae619 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:53.977401 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7da5d462-1597-4bd7-8176-eb46e3a02d4f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:53.977848 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7da5d462-1597-4bd7-8176-eb46e3a02d4f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:53.978609 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7da5d462-1597-4bd7-8176-eb46e3a02d4f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:53.978990 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-35766a78-5512-466f-b26d-bdd4831ae619 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:53.979094 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7da5d462-1597-4bd7-8176-eb46e3a02d4f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:53.979438 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-35766a78-5512-466f-b26d-bdd4831ae619 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:53.980196 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-35766a78-5512-466f-b26d-bdd4831ae619 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:53.980658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-35766a78-5512-466f-b26d-bdd4831ae619 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:54.169455 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-7da5d462-1597-4bd7-8176-eb46e3a02d4f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:54.176563 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-35766a78-5512-466f-b26d-bdd4831ae619 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:54.177012 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7da5d462-1597-4bd7-8176-eb46e3a02d4f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.210888 Jul 05 10:11:54.177582 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 124/287] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:53 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:54.184666 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-35766a78-5512-466f-b26d-bdd4831ae619 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.217020 Jul 05 10:11:54.185231 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 164/288] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:53 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:54.605872 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6762bdc6-e7e4-4fc1-9938-a16ddbe39fe5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:54.606358 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6762bdc6-e7e4-4fc1-9938-a16ddbe39fe5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:54.612458 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6762bdc6-e7e4-4fc1-9938-a16ddbe39fe5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:54.612905 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6762bdc6-e7e4-4fc1-9938-a16ddbe39fe5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:54.613654 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6762bdc6-e7e4-4fc1-9938-a16ddbe39fe5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:54.614430 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6762bdc6-e7e4-4fc1-9938-a16ddbe39fe5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:54.641655 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6762bdc6-e7e4-4fc1-9938-a16ddbe39fe5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Going to try to terminate instance {{(pid=117812) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:11:54.734517 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6762bdc6-e7e4-4fc1-9938-a16ddbe39fe5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 204 len: 0 microversion: 2.1 time: 0.132531 Jul 05 10:11:54.735132 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 125/289] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:11:54 2022] DELETE /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:11:54.743298 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-fea1685f-86ff-43c0-9099-e699497664d5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:54.743936 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-fea1685f-86ff-43c0-9099-e699497664d5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:54.750641 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fea1685f-86ff-43c0-9099-e699497664d5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:54.751091 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fea1685f-86ff-43c0-9099-e699497664d5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:54.751851 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fea1685f-86ff-43c0-9099-e699497664d5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:54.752477 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fea1685f-86ff-43c0-9099-e699497664d5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:54.909216 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-fea1685f-86ff-43c0-9099-e699497664d5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:54.924064 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-fea1685f-86ff-43c0-9099-e699497664d5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1554 microversion: 2.1 time: 0.181047 Jul 05 10:11:54.924064 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 165/290] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:54 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1554 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:55.194988 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-63a39bdb-a2e8-4097-aec0-f2a1f2d18a27 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:55.196449 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-63a39bdb-a2e8-4097-aec0-f2a1f2d18a27 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:55.200763 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6e9d249a-f54a-419e-862c-a8107df9f817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:55.201401 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-6e9d249a-f54a-419e-862c-a8107df9f817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:55.208750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6e9d249a-f54a-419e-862c-a8107df9f817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:55.209413 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6e9d249a-f54a-419e-862c-a8107df9f817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:55.210297 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-63a39bdb-a2e8-4097-aec0-f2a1f2d18a27 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:55.210456 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6e9d249a-f54a-419e-862c-a8107df9f817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:55.210754 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6e9d249a-f54a-419e-862c-a8107df9f817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:55.211369 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-63a39bdb-a2e8-4097-aec0-f2a1f2d18a27 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:55.212822 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-63a39bdb-a2e8-4097-aec0-f2a1f2d18a27 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:55.213413 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-63a39bdb-a2e8-4097-aec0-f2a1f2d18a27 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:55.421596 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6e9d249a-f54a-419e-862c-a8107df9f817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:55.425106 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-63a39bdb-a2e8-4097-aec0-f2a1f2d18a27 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:55.425369 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=100 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:55.430727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-6e9d249a-f54a-419e-862c-a8107df9f817 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.233821 Jul 05 10:11:55.431314 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 166/291] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:55 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:55.432674 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-63a39bdb-a2e8-4097-aec0-f2a1f2d18a27 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.242493 Jul 05 10:11:55.433279 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 126/292] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:55 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:55.942047 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-045edd2f-5e72-4e67-bf62-b53276bbd186 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:55.942621 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-045edd2f-5e72-4e67-bf62-b53276bbd186 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:55.948646 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-045edd2f-5e72-4e67-bf62-b53276bbd186 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:55.949664 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-045edd2f-5e72-4e67-bf62-b53276bbd186 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:55.950439 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-045edd2f-5e72-4e67-bf62-b53276bbd186 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:55.950877 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-045edd2f-5e72-4e67-bf62-b53276bbd186 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:56.099040 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-045edd2f-5e72-4e67-bf62-b53276bbd186 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:56.108738 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-045edd2f-5e72-4e67-bf62-b53276bbd186 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1554 microversion: 2.1 time: 0.170706 Jul 05 10:11:56.109313 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 167/293] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:55 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1554 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:56.447197 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-72815530-dd1d-4e9f-b7da-2fcb5258f0ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:56.448079 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-72815530-dd1d-4e9f-b7da-2fcb5258f0ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:56.449073 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d1d3f1c6-845d-406e-8f73-2b79cf6401ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:56.449868 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d1d3f1c6-845d-406e-8f73-2b79cf6401ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:56.456308 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72815530-dd1d-4e9f-b7da-2fcb5258f0ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:56.456461 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d1d3f1c6-845d-406e-8f73-2b79cf6401ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:56.456861 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72815530-dd1d-4e9f-b7da-2fcb5258f0ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:56.456987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d1d3f1c6-845d-406e-8f73-2b79cf6401ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:56.457626 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d1d3f1c6-845d-406e-8f73-2b79cf6401ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:56.457786 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72815530-dd1d-4e9f-b7da-2fcb5258f0ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:56.458062 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d1d3f1c6-845d-406e-8f73-2b79cf6401ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:56.458505 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72815530-dd1d-4e9f-b7da-2fcb5258f0ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:56.653273 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-72815530-dd1d-4e9f-b7da-2fcb5258f0ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:56.656410 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d1d3f1c6-845d-406e-8f73-2b79cf6401ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:56.660759 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-72815530-dd1d-4e9f-b7da-2fcb5258f0ff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.217507 Jul 05 10:11:56.661333 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 127/294] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:56 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:56.663831 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d1d3f1c6-845d-406e-8f73-2b79cf6401ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.218673 Jul 05 10:11:56.664421 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 168/295] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:56 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:57.125420 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-589d108b-ac5e-4887-b207-ee21560bf4da tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:57.126017 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-589d108b-ac5e-4887-b207-ee21560bf4da tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:57.132029 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-589d108b-ac5e-4887-b207-ee21560bf4da tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:57.132498 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-589d108b-ac5e-4887-b207-ee21560bf4da tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:57.133250 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-589d108b-ac5e-4887-b207-ee21560bf4da tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:57.133675 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-589d108b-ac5e-4887-b207-ee21560bf4da tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:57.260756 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-589d108b-ac5e-4887-b207-ee21560bf4da tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:57.269478 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-589d108b-ac5e-4887-b207-ee21560bf4da tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1554 microversion: 2.1 time: 0.147759 Jul 05 10:11:57.270049 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 128/296] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:57 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1554 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:57.678157 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-80a9d6c5-af1a-47e4-be86-4cd54e452aa2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:57.678774 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-80a9d6c5-af1a-47e4-be86-4cd54e452aa2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:57.681382 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-586ebd59-ba96-4af7-b772-82f37313833c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:57.681991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-586ebd59-ba96-4af7-b772-82f37313833c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:57.685705 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80a9d6c5-af1a-47e4-be86-4cd54e452aa2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:57.686162 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80a9d6c5-af1a-47e4-be86-4cd54e452aa2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:57.686929 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80a9d6c5-af1a-47e4-be86-4cd54e452aa2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:57.687371 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80a9d6c5-af1a-47e4-be86-4cd54e452aa2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:57.689022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-586ebd59-ba96-4af7-b772-82f37313833c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:57.689465 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-586ebd59-ba96-4af7-b772-82f37313833c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:57.690218 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-586ebd59-ba96-4af7-b772-82f37313833c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:57.690646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-586ebd59-ba96-4af7-b772-82f37313833c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:57.881073 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-80a9d6c5-af1a-47e4-be86-4cd54e452aa2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:57.884508 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-586ebd59-ba96-4af7-b772-82f37313833c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:57.888001 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:11:57.889143 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-80a9d6c5-af1a-47e4-be86-4cd54e452aa2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.214790 Jul 05 10:11:57.889909 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 169/297] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:57 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:57.893746 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-586ebd59-ba96-4af7-b772-82f37313833c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.216321 Jul 05 10:11:57.894350 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 129/298] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:57 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:11:58.286425 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b91156e5-c780-47e9-a3d3-1dea5d6bb01a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:58.287036 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b91156e5-c780-47e9-a3d3-1dea5d6bb01a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:58.294079 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b91156e5-c780-47e9-a3d3-1dea5d6bb01a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:58.294526 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b91156e5-c780-47e9-a3d3-1dea5d6bb01a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:58.295287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b91156e5-c780-47e9-a3d3-1dea5d6bb01a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:58.295725 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b91156e5-c780-47e9-a3d3-1dea5d6bb01a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:58.423509 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b91156e5-c780-47e9-a3d3-1dea5d6bb01a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:58.432802 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b91156e5-c780-47e9-a3d3-1dea5d6bb01a tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1554 microversion: 2.1 time: 0.149143 Jul 05 10:11:58.433452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 170/299] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:58 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1554 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:58.910104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f96b8f91-9962-4816-a331-960df75937b0 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:58.910709 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f96b8f91-9962-4816-a331-960df75937b0 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:58.912310 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5fc170f0-fa0d-4753-91c6-bbb0f6981e4f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:58.912945 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5fc170f0-fa0d-4753-91c6-bbb0f6981e4f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:58.918967 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f96b8f91-9962-4816-a331-960df75937b0 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:58.919138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5fc170f0-fa0d-4753-91c6-bbb0f6981e4f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:58.919421 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5fc170f0-fa0d-4753-91c6-bbb0f6981e4f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:58.919911 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f96b8f91-9962-4816-a331-960df75937b0 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:58.920193 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5fc170f0-fa0d-4753-91c6-bbb0f6981e4f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:58.920651 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5fc170f0-fa0d-4753-91c6-bbb0f6981e4f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:58.921033 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f96b8f91-9962-4816-a331-960df75937b0 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:58.921472 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f96b8f91-9962-4816-a331-960df75937b0 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:59.119140 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f96b8f91-9962-4816-a331-960df75937b0 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:59.122661 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5fc170f0-fa0d-4753-91c6-bbb0f6981e4f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:59.127660 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f96b8f91-9962-4816-a331-960df75937b0 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.220875 Jul 05 10:11:59.127759 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 130/300] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:58 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:59.130311 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5fc170f0-fa0d-4753-91c6-bbb0f6981e4f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.221791 Jul 05 10:11:59.131023 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 171/301] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:58 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:11:59.449515 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-48e71af5-7b03-4517-baf2-56b5fde5fbc3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:11:59.450118 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-48e71af5-7b03-4517-baf2-56b5fde5fbc3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:11:59.456283 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-48e71af5-7b03-4517-baf2-56b5fde5fbc3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:59.456762 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-48e71af5-7b03-4517-baf2-56b5fde5fbc3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:59.457520 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-48e71af5-7b03-4517-baf2-56b5fde5fbc3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:11:59.457958 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-48e71af5-7b03-4517-baf2-56b5fde5fbc3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:11:59.584436 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-48e71af5-7b03-4517-baf2-56b5fde5fbc3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:11:59.592752 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-48e71af5-7b03-4517-baf2-56b5fde5fbc3 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1554 microversion: 2.1 time: 0.146999 Jul 05 10:11:59.593320 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 131/302] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:11:59 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1554 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:00.143691 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-05f5f649-5a1b-4b5d-bcf9-c38de542aa58 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:00.144316 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-05f5f649-5a1b-4b5d-bcf9-c38de542aa58 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:00.146925 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-11e16e4d-8106-4d61-80a4-38e592d2dc41 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:00.147530 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-11e16e4d-8106-4d61-80a4-38e592d2dc41 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:00.150267 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-05f5f649-5a1b-4b5d-bcf9-c38de542aa58 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:00.150705 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-05f5f649-5a1b-4b5d-bcf9-c38de542aa58 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:00.151452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-05f5f649-5a1b-4b5d-bcf9-c38de542aa58 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:00.151879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-05f5f649-5a1b-4b5d-bcf9-c38de542aa58 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:00.154442 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-11e16e4d-8106-4d61-80a4-38e592d2dc41 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:00.154888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-11e16e4d-8106-4d61-80a4-38e592d2dc41 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:00.155648 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-11e16e4d-8106-4d61-80a4-38e592d2dc41 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:00.156088 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-11e16e4d-8106-4d61-80a4-38e592d2dc41 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:00.349768 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-05f5f649-5a1b-4b5d-bcf9-c38de542aa58 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:00.352992 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-11e16e4d-8106-4d61-80a4-38e592d2dc41 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:00.360868 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-05f5f649-5a1b-4b5d-bcf9-c38de542aa58 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1314 microversion: 2.1 time: 0.220945 Jul 05 10:12:00.361071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-11e16e4d-8106-4d61-80a4-38e592d2dc41 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.217854 Jul 05 10:12:00.368605 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 132/304] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:00 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:00.368930 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 172/304] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:00 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1314 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:00.610946 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-75eee912-2134-4522-a5c2-50154397183e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:00.611562 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-75eee912-2134-4522-a5c2-50154397183e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:00.617698 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-75eee912-2134-4522-a5c2-50154397183e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:00.618163 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-75eee912-2134-4522-a5c2-50154397183e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:00.618918 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-75eee912-2134-4522-a5c2-50154397183e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:00.619353 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-75eee912-2134-4522-a5c2-50154397183e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:00.761083 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-75eee912-2134-4522-a5c2-50154397183e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:00.769316 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-75eee912-2134-4522-a5c2-50154397183e tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1554 microversion: 2.1 time: 0.162777 Jul 05 10:12:00.769821 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 133/305] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:00 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1554 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:01.378271 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c6a8cdce-c2a2-4d27-9722-a2a50b9a2fa7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:01.378990 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c6a8cdce-c2a2-4d27-9722-a2a50b9a2fa7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:01.383860 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0de50b78-eaa8-4954-a565-327fcaeed854 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:01.384508 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0de50b78-eaa8-4954-a565-327fcaeed854 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:01.385081 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c6a8cdce-c2a2-4d27-9722-a2a50b9a2fa7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:01.385534 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c6a8cdce-c2a2-4d27-9722-a2a50b9a2fa7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:01.386300 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c6a8cdce-c2a2-4d27-9722-a2a50b9a2fa7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:01.386781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c6a8cdce-c2a2-4d27-9722-a2a50b9a2fa7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:01.390941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0de50b78-eaa8-4954-a565-327fcaeed854 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:01.391401 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0de50b78-eaa8-4954-a565-327fcaeed854 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:01.392170 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0de50b78-eaa8-4954-a565-327fcaeed854 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:01.392633 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0de50b78-eaa8-4954-a565-327fcaeed854 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:01.583747 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c6a8cdce-c2a2-4d27-9722-a2a50b9a2fa7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:01.587969 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-0de50b78-eaa8-4954-a565-327fcaeed854 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:01.591462 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c6a8cdce-c2a2-4d27-9722-a2a50b9a2fa7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.217115 Jul 05 10:12:01.592510 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 173/306] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:01 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:01.607838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0de50b78-eaa8-4954-a565-327fcaeed854 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1499 microversion: 2.1 time: 0.227730 Jul 05 10:12:01.608454 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 134/307] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:01 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1499 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:01.787704 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5260cddc-be47-4426-8803-ded705e00464 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:01.788803 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5260cddc-be47-4426-8803-ded705e00464 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:01.799616 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5260cddc-be47-4426-8803-ded705e00464 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:01.800065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5260cddc-be47-4426-8803-ded705e00464 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:01.800979 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5260cddc-be47-4426-8803-ded705e00464 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:01.801611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5260cddc-be47-4426-8803-ded705e00464 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:01.965288 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5260cddc-be47-4426-8803-ded705e00464 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:01.973400 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5260cddc-be47-4426-8803-ded705e00464 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1554 microversion: 2.1 time: 0.190026 Jul 05 10:12:01.974249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 174/308] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:01 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1554 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:02.382129 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ba2d2e03-3375-4bbb-a1f9-c87ddc3aea4a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:02.382622 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ba2d2e03-3375-4bbb-a1f9-c87ddc3aea4a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:02.392015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ba2d2e03-3375-4bbb-a1f9-c87ddc3aea4a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:02.392682 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ba2d2e03-3375-4bbb-a1f9-c87ddc3aea4a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:02.393284 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ba2d2e03-3375-4bbb-a1f9-c87ddc3aea4a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:02.393720 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ba2d2e03-3375-4bbb-a1f9-c87ddc3aea4a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:02.420715 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ba2d2e03-3375-4bbb-a1f9-c87ddc3aea4a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Going to try to terminate instance {{(pid=117812) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:12:02.534382 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ba2d2e03-3375-4bbb-a1f9-c87ddc3aea4a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "DELETE /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 204 len: 0 microversion: 2.1 time: 0.154821 Jul 05 10:12:02.535193 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 135/309] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:12:02 2022] DELETE /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:12:02.546931 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-391cd69d-b184-4448-9f60-35386b682db5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:02.547547 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-391cd69d-b184-4448-9f60-35386b682db5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:02.554981 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-391cd69d-b184-4448-9f60-35386b682db5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:02.555662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-391cd69d-b184-4448-9f60-35386b682db5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:02.556778 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-391cd69d-b184-4448-9f60-35386b682db5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:02.557512 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-391cd69d-b184-4448-9f60-35386b682db5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:02.610814 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-27188c9a-fdc5-4bed-9bf5-3cc7de637418 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:02.611675 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-27188c9a-fdc5-4bed-9bf5-3cc7de637418 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:02.626925 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-27188c9a-fdc5-4bed-9bf5-3cc7de637418 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:02.627379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-27188c9a-fdc5-4bed-9bf5-3cc7de637418 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:02.628169 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-27188c9a-fdc5-4bed-9bf5-3cc7de637418 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:02.630340 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-27188c9a-fdc5-4bed-9bf5-3cc7de637418 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:02.908752 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-391cd69d-b184-4448-9f60-35386b682db5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:02.922071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-391cd69d-b184-4448-9f60-35386b682db5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1505 microversion: 2.1 time: 0.378944 Jul 05 10:12:02.923127 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 175/310] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:02 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1505 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:02.999992 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-967cab1f-3214-4f17-a044-20896a2187f4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:03.001150 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-967cab1f-3214-4f17-a044-20896a2187f4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:03.012489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-967cab1f-3214-4f17-a044-20896a2187f4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:03.012489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-967cab1f-3214-4f17-a044-20896a2187f4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:03.012489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-967cab1f-3214-4f17-a044-20896a2187f4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:03.012489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-967cab1f-3214-4f17-a044-20896a2187f4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:03.027000 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-27188c9a-fdc5-4bed-9bf5-3cc7de637418 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:03.065939 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-27188c9a-fdc5-4bed-9bf5-3cc7de637418 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.459212 Jul 05 10:12:03.067858 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 136/311] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:02 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:03.309180 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-85d461e9-ca82-4559-a302-690462f2a110 req-f46c942d-bd7c-42b4-843b-608353608b0a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b7ce1e24-0c58-4071-9610-57d072131857", "name": "network-vif-deleted", "tag": "1590f070-4b3b-4762-9ba6-291ca1720666"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:12:03.321542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-85d461e9-ca82-4559-a302-690462f2a110 req-f46c942d-bd7c-42b4-843b-608353608b0a service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:03.321997 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-85d461e9-ca82-4559-a302-690462f2a110 req-f46c942d-bd7c-42b4-843b-608353608b0a service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:03.358147 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-85d461e9-ca82-4559-a302-690462f2a110 req-f46c942d-bd7c-42b4-843b-608353608b0a service nova] Creating event network-vif-deleted:1590f070-4b3b-4762-9ba6-291ca1720666 for instance b7ce1e24-0c58-4071-9610-57d072131857 on hv-cinder-maste Jul 05 10:12:03.369919 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-85d461e9-ca82-4559-a302-690462f2a110 req-f46c942d-bd7c-42b4-843b-608353608b0a service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064740 Jul 05 10:12:03.372506 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 137/312] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:12:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:12:03.389571 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-967cab1f-3214-4f17-a044-20896a2187f4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:03.398742 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-967cab1f-3214-4f17-a044-20896a2187f4 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1512 microversion: 2.1 time: 0.409262 Jul 05 10:12:03.412680 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 176/313] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:02 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1512 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:03.942058 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0364454a-adeb-4489-9cbd-5c1353fb9d91 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:03.942874 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0364454a-adeb-4489-9cbd-5c1353fb9d91 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:03.950223 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0364454a-adeb-4489-9cbd-5c1353fb9d91 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:03.950223 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0364454a-adeb-4489-9cbd-5c1353fb9d91 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:03.951089 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0364454a-adeb-4489-9cbd-5c1353fb9d91 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:03.952187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0364454a-adeb-4489-9cbd-5c1353fb9d91 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:04.086274 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5782bd21-de42-4bd3-a71a-6ded583b81e6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:04.087522 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5782bd21-de42-4bd3-a71a-6ded583b81e6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:04.097291 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5782bd21-de42-4bd3-a71a-6ded583b81e6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:04.097836 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5782bd21-de42-4bd3-a71a-6ded583b81e6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:04.098681 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5782bd21-de42-4bd3-a71a-6ded583b81e6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:04.099325 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5782bd21-de42-4bd3-a71a-6ded583b81e6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:04.155965 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-0364454a-adeb-4489-9cbd-5c1353fb9d91 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:04.165033 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0364454a-adeb-4489-9cbd-5c1353fb9d91 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1505 microversion: 2.1 time: 0.227492 Jul 05 10:12:04.165609 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 138/314] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:03 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1505 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:04.308865 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5782bd21-de42-4bd3-a71a-6ded583b81e6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:04.316993 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5782bd21-de42-4bd3-a71a-6ded583b81e6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.234742 Jul 05 10:12:04.317806 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 177/315] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:04 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:04.417383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d22b9ad1-bfb2-4d60-89aa-11d05f913b7d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:04.418233 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d22b9ad1-bfb2-4d60-89aa-11d05f913b7d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:04.424585 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d22b9ad1-bfb2-4d60-89aa-11d05f913b7d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:04.425321 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d22b9ad1-bfb2-4d60-89aa-11d05f913b7d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:04.426133 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d22b9ad1-bfb2-4d60-89aa-11d05f913b7d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:04.426772 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d22b9ad1-bfb2-4d60-89aa-11d05f913b7d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:04.555485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d22b9ad1-bfb2-4d60-89aa-11d05f913b7d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9d95652b0714a2a8427a7e99de27bee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1f4fee9c6b844a2b7a14e2afb831f88', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:04.568960 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d22b9ad1-bfb2-4d60-89aa-11d05f913b7d tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 200 len: 1346 microversion: 2.1 time: 0.155617 Jul 05 10:12:04.569701 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 139/316] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:04 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 1346 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:05.183613 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a394b96b-d7a3-4c5b-8a36-4f0953e4c86f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:05.185443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a394b96b-d7a3-4c5b-8a36-4f0953e4c86f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:05.193931 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a394b96b-d7a3-4c5b-8a36-4f0953e4c86f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:05.194561 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a394b96b-d7a3-4c5b-8a36-4f0953e4c86f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:05.195501 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a394b96b-d7a3-4c5b-8a36-4f0953e4c86f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:05.195934 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a394b96b-d7a3-4c5b-8a36-4f0953e4c86f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:05.336617 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0d51a6cf-ffa2-440e-99a4-47834f8d4d1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:05.339314 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0d51a6cf-ffa2-440e-99a4-47834f8d4d1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:05.347796 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0d51a6cf-ffa2-440e-99a4-47834f8d4d1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:05.348573 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0d51a6cf-ffa2-440e-99a4-47834f8d4d1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:05.349576 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0d51a6cf-ffa2-440e-99a4-47834f8d4d1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:05.350260 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0d51a6cf-ffa2-440e-99a4-47834f8d4d1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:05.525608 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a394b96b-d7a3-4c5b-8a36-4f0953e4c86f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:05.545284 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a394b96b-d7a3-4c5b-8a36-4f0953e4c86f tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1505 microversion: 2.1 time: 0.364195 Jul 05 10:12:05.545893 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 178/317] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:05 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1505 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:05.588589 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-cce476c8-3508-45fc-ae1e-c8ab23af3dc5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:05.589522 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-cce476c8-3508-45fc-ae1e-c8ab23af3dc5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:05.596537 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cce476c8-3508-45fc-ae1e-c8ab23af3dc5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:05.597651 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cce476c8-3508-45fc-ae1e-c8ab23af3dc5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:05.598588 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cce476c8-3508-45fc-ae1e-c8ab23af3dc5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:05.599221 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cce476c8-3508-45fc-ae1e-c8ab23af3dc5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:05.617696 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-cce476c8-3508-45fc-ae1e-c8ab23af3dc5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] HTTP exception thrown: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-cce476c8-3508-45fc-ae1e-c8ab23af3dc5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Request method failure captured: Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 HTTP/1.1 Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4FF5e3BTWIk8iPSdQNaXTymUZPWar3RWVWpUJbKtbtWl2UHeh0_U7pyyad2h9IovKQhmPRo39q_-8WGDQ24Bg9arEZS5dwgammqb8XSngxfTvuhDw3I9ybpbulZPhHLawCMRFqR-jA_kS-7dw0Ia7ecAVYPE0TTFNuecWBECoeEyA Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: f1f4fee9c6b844a2b7a14e2afb831f88 Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-DeleteServersTestJSON-690662559 Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:12:05.623125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:12:05.624674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f1f4fee9c6b844a2b7a14e2afb831f88"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f1f4fee9c6b844a2b7a14e2afb831f88"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f1f4fee9c6b844a2b7a14e2afb831f88"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f1f4fee9c6b844a2b7a14e2afb831f88"}]}] Jul 05 10:12:05.624674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-DeleteServersTestJSON-690662559 Jul 05 10:12:05.624674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: f1f4fee9c6b844a2b7a14e2afb831f88 Jul 05 10:12:05.624674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-DeleteServersTestJSON-690662559 Jul 05 10:12:05.624674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-DeleteServersTestJSON-690662559-project Jul 05 10:12:05.624674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:12:05.624674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:12:05.624674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: a9d95652b0714a2a8427a7e99de27bee Jul 05 10:12:05.624674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-DeleteServersTestJSON-690662559-project Jul 05 10:12:05.624674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:12:05.624674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:05.624674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'b7ce1e24-0c58-4071-9610-57d072131857'} Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:12:05.625545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:05.627078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:12:05.629210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-cce476c8-3508-45fc-ae1e-c8ab23af3dc5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Returning 404 to user: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:12:05.629210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-cce476c8-3508-45fc-ae1e-c8ab23af3dc5 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 404 len: 111 microversion: 2.1 time: 0.041539 Jul 05 10:12:05.629210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 179/318] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:05 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 05 10:12:05.701522 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-f950e06d-fd79-484c-8276-925567b514a6 req-a301a859-d71d-4208-b60a-872f5d3e959f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbea07d3-b6f1-479e-9fcb-abcfa716bb5b", "name": "network-vif-unplugged", "status": "completed", "tag": "4e423f85-def7-4513-9e8f-09149af663a9"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:12:05.727963 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-0d51a6cf-ffa2-440e-99a4-47834f8d4d1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:05.737476 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-f950e06d-fd79-484c-8276-925567b514a6 req-a301a859-d71d-4208-b60a-872f5d3e959f service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:05.738158 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0d51a6cf-ffa2-440e-99a4-47834f8d4d1a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.406447 Jul 05 10:12:05.738544 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-f950e06d-fd79-484c-8276-925567b514a6 req-a301a859-d71d-4208-b60a-872f5d3e959f service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:05.738734 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 140/319] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:05 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:05.761571 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-f950e06d-fd79-484c-8276-925567b514a6 req-a301a859-d71d-4208-b60a-872f5d3e959f service nova] Creating event network-vif-unplugged:4e423f85-def7-4513-9e8f-09149af663a9 for instance fbea07d3-b6f1-479e-9fcb-abcfa716bb5b on hv-cinder-maste Jul 05 10:12:05.768160 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-f950e06d-fd79-484c-8276-925567b514a6 req-a301a859-d71d-4208-b60a-872f5d3e959f service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.074907 Jul 05 10:12:05.769639 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 180/320] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:12:05 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:12:05.822895 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-9f96c2c1-de00-4da8-84e8-08e395aded3c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:05.823887 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-9f96c2c1-de00-4da8-84e8-08e395aded3c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:05.830336 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9f96c2c1-de00-4da8-84e8-08e395aded3c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:05.831025 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9f96c2c1-de00-4da8-84e8-08e395aded3c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:05.832040 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9f96c2c1-de00-4da8-84e8-08e395aded3c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:05.832648 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9f96c2c1-de00-4da8-84e8-08e395aded3c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:05.849395 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-9f96c2c1-de00-4da8-84e8-08e395aded3c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] HTTP exception thrown: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-9f96c2c1-de00-4da8-84e8-08e395aded3c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Request method failure captured: Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857/os-volume_attachments HTTP/1.1 Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA4FF5e3BTWIk8iPSdQNaXTymUZPWar3RWVWpUJbKtbtWl2UHeh0_U7pyyad2h9IovKQhmPRo39q_-8WGDQ24Bg9arEZS5dwgammqb8XSngxfTvuhDw3I9ybpbulZPhHLawCMRFqR-jA_kS-7dw0Ia7ecAVYPE0TTFNuecWBECoeEyA Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: f1f4fee9c6b844a2b7a14e2afb831f88 Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-DeleteServersTestJSON-690662559 Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:12:05.853931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:12:05.855503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f1f4fee9c6b844a2b7a14e2afb831f88"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f1f4fee9c6b844a2b7a14e2afb831f88"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f1f4fee9c6b844a2b7a14e2afb831f88"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f1f4fee9c6b844a2b7a14e2afb831f88"}]}] Jul 05 10:12:05.855503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-DeleteServersTestJSON-690662559 Jul 05 10:12:05.855503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: f1f4fee9c6b844a2b7a14e2afb831f88 Jul 05 10:12:05.855503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-DeleteServersTestJSON-690662559 Jul 05 10:12:05.855503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-DeleteServersTestJSON-690662559-project Jul 05 10:12:05.855503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:12:05.855503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:12:05.855503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: a9d95652b0714a2a8427a7e99de27bee Jul 05 10:12:05.855503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-DeleteServersTestJSON-690662559-project Jul 05 10:12:05.855503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:12:05.855503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:05.855503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'server_id': 'b7ce1e24-0c58-4071-9610-57d072131857'} Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:12:05.857022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:05.858375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:05.860657 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-9f96c2c1-de00-4da8-84e8-08e395aded3c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Returning 404 to user: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:12:05.861695 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-9f96c2c1-de00-4da8-84e8-08e395aded3c tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.042708 Jul 05 10:12:05.863126 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 141/321] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:12:05 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857/os-volume_attachments => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:12:06.169883 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-cf75356b-ea10-43ef-88a4-74422e5302af req-09e986cc-f30c-4d29-96ab-a34a7622a6b4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbea07d3-b6f1-479e-9fcb-abcfa716bb5b", "name": "network-vif-deleted", "tag": "4e423f85-def7-4513-9e8f-09149af663a9"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:12:06.186206 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-cf75356b-ea10-43ef-88a4-74422e5302af req-09e986cc-f30c-4d29-96ab-a34a7622a6b4 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:06.186767 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-cf75356b-ea10-43ef-88a4-74422e5302af req-09e986cc-f30c-4d29-96ab-a34a7622a6b4 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:06.202560 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-cf75356b-ea10-43ef-88a4-74422e5302af req-09e986cc-f30c-4d29-96ab-a34a7622a6b4 service nova] Creating event network-vif-deleted:4e423f85-def7-4513-9e8f-09149af663a9 for instance fbea07d3-b6f1-479e-9fcb-abcfa716bb5b on hv-cinder-maste Jul 05 10:12:06.207846 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-cf75356b-ea10-43ef-88a4-74422e5302af req-09e986cc-f30c-4d29-96ab-a34a7622a6b4 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045881 Jul 05 10:12:06.208748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 181/322] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:12:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:12:06.563606 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-674fb65a-b138-4dd7-bf81-f8b816e8df01 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:06.564578 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-674fb65a-b138-4dd7-bf81-f8b816e8df01 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:06.571721 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-674fb65a-b138-4dd7-bf81-f8b816e8df01 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:06.572863 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-674fb65a-b138-4dd7-bf81-f8b816e8df01 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:06.573882 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-674fb65a-b138-4dd7-bf81-f8b816e8df01 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:06.574497 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-674fb65a-b138-4dd7-bf81-f8b816e8df01 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:06.615014 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-674fb65a-b138-4dd7-bf81-f8b816e8df01 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:06.615628 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-674fb65a-b138-4dd7-bf81-f8b816e8df01 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:06.616159 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-674fb65a-b138-4dd7-bf81-f8b816e8df01 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lazy-loading 'fault' on Instance uuid fbea07d3-b6f1-479e-9fcb-abcfa716bb5b {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:12:06.690664 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-674fb65a-b138-4dd7-bf81-f8b816e8df01 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:06.698195 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-674fb65a-b138-4dd7-bf81-f8b816e8df01 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 200 len: 1304 microversion: 2.1 time: 0.138846 Jul 05 10:12:06.698888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 142/323] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:06 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 1304 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:06.755737 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0a018b0c-b42d-4dd3-9bfe-352f302feb51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:06.756452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0a018b0c-b42d-4dd3-9bfe-352f302feb51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:06.763452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0a018b0c-b42d-4dd3-9bfe-352f302feb51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:06.763894 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0a018b0c-b42d-4dd3-9bfe-352f302feb51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:06.764673 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0a018b0c-b42d-4dd3-9bfe-352f302feb51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:06.765115 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0a018b0c-b42d-4dd3-9bfe-352f302feb51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:07.027720 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0a018b0c-b42d-4dd3-9bfe-352f302feb51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:07.036977 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0a018b0c-b42d-4dd3-9bfe-352f302feb51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.285340 Jul 05 10:12:07.037559 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 182/324] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:06 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:07.141372 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-70f8ec74-e852-4a99-96e8-5193f0a159a9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:07.141884 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-70f8ec74-e852-4a99-96e8-5193f0a159a9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:07.155080 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-70f8ec74-e852-4a99-96e8-5193f0a159a9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:07.157054 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-70f8ec74-e852-4a99-96e8-5193f0a159a9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:07.158169 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-70f8ec74-e852-4a99-96e8-5193f0a159a9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:07.159117 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-70f8ec74-e852-4a99-96e8-5193f0a159a9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-70f8ec74-e852-4a99-96e8-5193f0a159a9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] HTTP exception thrown: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-70f8ec74-e852-4a99-96e8-5193f0a159a9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Request method failure captured: Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: DELETE /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 HTTP/1.1 Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA4FF5e3BTWIk8iPSdQNaXTymUZPWar3RWVWpUJbKtbtWl2UHeh0_U7pyyad2h9IovKQhmPRo39q_-8WGDQ24Bg9arEZS5dwgammqb8XSngxfTvuhDw3I9ybpbulZPhHLawCMRFqR-jA_kS-7dw0Ia7ecAVYPE0TTFNuecWBECoeEyA Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: f1f4fee9c6b844a2b7a14e2afb831f88 Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-DeleteServersTestJSON-690662559 Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:12:07.207007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:12:07.208537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f1f4fee9c6b844a2b7a14e2afb831f88"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f1f4fee9c6b844a2b7a14e2afb831f88"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f1f4fee9c6b844a2b7a14e2afb831f88"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f1f4fee9c6b844a2b7a14e2afb831f88"}]}] Jul 05 10:12:07.208537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-DeleteServersTestJSON-690662559 Jul 05 10:12:07.208537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: f1f4fee9c6b844a2b7a14e2afb831f88 Jul 05 10:12:07.208537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-DeleteServersTestJSON-690662559 Jul 05 10:12:07.208537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-DeleteServersTestJSON-690662559-project Jul 05 10:12:07.208537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:12:07.208537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:12:07.208537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: a9d95652b0714a2a8427a7e99de27bee Jul 05 10:12:07.208537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-DeleteServersTestJSON-690662559-project Jul 05 10:12:07.208537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:12:07.208537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:07.208537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': 'b7ce1e24-0c58-4071-9610-57d072131857'} Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:12:07.209403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:07.210646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.212002 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-70f8ec74-e852-4a99-96e8-5193f0a159a9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Returning 404 to user: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:12:07.212002 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-70f8ec74-e852-4a99-96e8-5193f0a159a9 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 404 len: 111 microversion: 2.1 time: 0.068171 Jul 05 10:12:07.212002 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 143/325] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:12:07 2022] DELETE /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:12:07.224116 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9cc390d8-21a4-4d1e-84f0-285d9657fca6 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:07.224887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9cc390d8-21a4-4d1e-84f0-285d9657fca6 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] [instance: b7ce1e24-0c58-4071-9610-57d072131857] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:07.240737 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9cc390d8-21a4-4d1e-84f0-285d9657fca6 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:07.241426 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9cc390d8-21a4-4d1e-84f0-285d9657fca6 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:07.243254 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9cc390d8-21a4-4d1e-84f0-285d9657fca6 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:07.243694 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9cc390d8-21a4-4d1e-84f0-285d9657fca6 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:07.269635 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-9cc390d8-21a4-4d1e-84f0-285d9657fca6 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] HTTP exception thrown: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9cc390d8-21a4-4d1e-84f0-285d9657fca6 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Request method failure captured: Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 HTTP/1.1 Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4FF5e3BTWIk8iPSdQNaXTymUZPWar3RWVWpUJbKtbtWl2UHeh0_U7pyyad2h9IovKQhmPRo39q_-8WGDQ24Bg9arEZS5dwgammqb8XSngxfTvuhDw3I9ybpbulZPhHLawCMRFqR-jA_kS-7dw0Ia7ecAVYPE0TTFNuecWBECoeEyA Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: f1f4fee9c6b844a2b7a14e2afb831f88 Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-DeleteServersTestJSON-690662559 Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:12:07.271484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:12:07.272760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f1f4fee9c6b844a2b7a14e2afb831f88"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f1f4fee9c6b844a2b7a14e2afb831f88"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f1f4fee9c6b844a2b7a14e2afb831f88"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f1f4fee9c6b844a2b7a14e2afb831f88"}]}] Jul 05 10:12:07.272760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-DeleteServersTestJSON-690662559 Jul 05 10:12:07.272760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: f1f4fee9c6b844a2b7a14e2afb831f88 Jul 05 10:12:07.272760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-DeleteServersTestJSON-690662559 Jul 05 10:12:07.272760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-DeleteServersTestJSON-690662559-project Jul 05 10:12:07.272760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:12:07.272760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:12:07.272760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: a9d95652b0714a2a8427a7e99de27bee Jul 05 10:12:07.272760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-DeleteServersTestJSON-690662559-project Jul 05 10:12:07.272760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:12:07.272760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:07.272760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'b7ce1e24-0c58-4071-9610-57d072131857'} Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:12:07.274092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. Jul 05 10:12:07.279381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.280571 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9cc390d8-21a4-4d1e-84f0-285d9657fca6 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] Returning 404 to user: Instance b7ce1e24-0c58-4071-9610-57d072131857 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:12:07.280571 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9cc390d8-21a4-4d1e-84f0-285d9657fca6 tempest-DeleteServersTestJSON-690662559 tempest-DeleteServersTestJSON-690662559-project] 10.222.0.20 "GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857" status: 404 len: 111 microversion: 2.1 time: 0.058732 Jul 05 10:12:07.280571 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 183/326] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:07 2022] GET /compute/v2.1/servers/b7ce1e24-0c58-4071-9610-57d072131857 => generated 111 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:12:07.718073 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-37c15b02-68ac-4949-9415-0c0a4194254a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:07.719095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-37c15b02-68ac-4949-9415-0c0a4194254a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: fbea07d3-b6f1-479e-9fcb-abcfa716bb5b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:07.726458 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-37c15b02-68ac-4949-9415-0c0a4194254a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:07.728313 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-37c15b02-68ac-4949-9415-0c0a4194254a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:07.729953 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-37c15b02-68ac-4949-9415-0c0a4194254a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:07.731308 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-37c15b02-68ac-4949-9415-0c0a4194254a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:07.753751 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-37c15b02-68ac-4949-9415-0c0a4194254a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] HTTP exception thrown: Instance fbea07d3-b6f1-479e-9fcb-abcfa716bb5b could not be found. Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-37c15b02-68ac-4949-9415-0c0a4194254a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Request method failure captured: Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b HTTP/1.1 Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA4nwoqtNHVEvvZjhlsgccGBfHw_4GX7-bJJaYYcYiHLm2podokcNX5Ngzj_K06wB_xDRq09afqP_YQKU63PBkD-_XweEXGvhD1Dj4YQljtZfjnDCsFoKAy3jWCECVjLpwSdQjvsbXx0wRW_ol9Kz--_dhmWVjih19FrCu8OSrTb8hA Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: c48f677354594fdc839367fff8b3dc29 Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-VolumesActionsTest-28417478 Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:12:07.757156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:12:07.758879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/c48f677354594fdc839367fff8b3dc29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_c48f677354594fdc839367fff8b3dc29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/c48f677354594fdc839367fff8b3dc29"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/c48f677354594fdc839367fff8b3dc29"}]}] Jul 05 10:12:07.758879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-VolumesActionsTest-28417478 Jul 05 10:12:07.758879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: c48f677354594fdc839367fff8b3dc29 Jul 05 10:12:07.758879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-VolumesActionsTest-28417478 Jul 05 10:12:07.758879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-VolumesActionsTest-28417478-project Jul 05 10:12:07.758879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:12:07.758879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:12:07.758879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: d2d25be9c4674aa09b2975f1c6bc8c52 Jul 05 10:12:07.758879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-VolumesActionsTest-28417478-project Jul 05 10:12:07.758879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:12:07.758879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance fbea07d3-b6f1-479e-9fcb-abcfa716bb5b could not be found. Jul 05 10:12:07.758879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': 'fbea07d3-b6f1-479e-9fcb-abcfa716bb5b'} Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbea07d3-b6f1-479e-9fcb-abcfa716bb5b could not be found. Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:12:07.759670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbea07d3-b6f1-479e-9fcb-abcfa716bb5b could not be found. Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbea07d3-b6f1-479e-9fcb-abcfa716bb5b could not be found. Jul 05 10:12:07.760996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:07.762255 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-37c15b02-68ac-4949-9415-0c0a4194254a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Returning 404 to user: Instance fbea07d3-b6f1-479e-9fcb-abcfa716bb5b could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:12:07.763601 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-37c15b02-68ac-4949-9415-0c0a4194254a tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b" status: 404 len: 111 microversion: 2.1 time: 0.050381 Jul 05 10:12:07.765328 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 144/327] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:07 2022] GET /compute/v2.1/servers/fbea07d3-b6f1-479e-9fcb-abcfa716bb5b => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:12:07.891975 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-417115549", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "networks": [{"uuid": "47de8f46-bb2e-4830-b096-bb218c860b15"}]}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:12:07.964306 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.network.neutron [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] validate_networks() for [('47de8f46-bb2e-4830-b096-bb218c860b15', None, None, None, None, None)] {{(pid=117813) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:12:08.054592 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3ad596d1-182a-4168-80da-8ab5272fdbc0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:08.055110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3ad596d1-182a-4168-80da-8ab5272fdbc0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:08.061321 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3ad596d1-182a-4168-80da-8ab5272fdbc0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:08.061763 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3ad596d1-182a-4168-80da-8ab5272fdbc0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:08.062712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3ad596d1-182a-4168-80da-8ab5272fdbc0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:08.063149 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3ad596d1-182a-4168-80da-8ab5272fdbc0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:08.445502 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-3ad596d1-182a-4168-80da-8ab5272fdbc0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:08.454911 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3ad596d1-182a-4168-80da-8ab5272fdbc0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.404417 Jul 05 10:12:08.469852 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 145/328] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:08 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:08.572663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Flavor limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:12:08.573793 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Image limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:12:08.574679 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Flavor pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:12:08.575713 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Image pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:12:08.576446 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:12:08.587870 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:08.589108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:08.590882 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:08.591789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:08.623133 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Getting quotas for project c48f677354594fdc839367fff8b3dc29. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:12:08.629763 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Getting quotas for user d2d25be9c4674aa09b2975f1c6bc8c52 and project c48f677354594fdc839367fff8b3dc29. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:12:08.649201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Going to run 1 instances... {{(pid=117813) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:12:08.652182 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117813) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:12:08.714754 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5a338969-09da-405e-9b32-9e4b86cca666 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.828924 Jul 05 10:12:08.716609 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 184/329] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:12:07 2022] POST /compute/v2.1/servers => generated 380 bytes in 832 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:12:08.728297 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-4737a6b0-72ce-4a41-9f8e-e754f09ab5b9 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:08.729187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-4737a6b0-72ce-4a41-9f8e-e754f09ab5b9 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:08.879178 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-4737a6b0-72ce-4a41-9f8e-e754f09ab5b9 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:08.886547 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-4737a6b0-72ce-4a41-9f8e-e754f09ab5b9 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1213 microversion: 2.1 time: 0.163685 Jul 05 10:12:08.887972 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 146/330] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:08 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1213 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:09.480787 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a3736561-faf5-45ff-b9dd-09ee3b548f01 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:09.481931 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a3736561-faf5-45ff-b9dd-09ee3b548f01 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:09.494948 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:09.505425 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a3736561-faf5-45ff-b9dd-09ee3b548f01 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:09.506258 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a3736561-faf5-45ff-b9dd-09ee3b548f01 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:09.511783 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a3736561-faf5-45ff-b9dd-09ee3b548f01 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:09.514041 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a3736561-faf5-45ff-b9dd-09ee3b548f01 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:09.885477 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a3736561-faf5-45ff-b9dd-09ee3b548f01 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:09.905131 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d1485589-9260-4d31-90a8-7479d1a1713e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:09.906197 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d1485589-9260-4d31-90a8-7479d1a1713e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:09.913703 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a3736561-faf5-45ff-b9dd-09ee3b548f01 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.444805 Jul 05 10:12:09.917454 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 185/331] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:09 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:09.941679 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d1485589-9260-4d31-90a8-7479d1a1713e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:09.941832 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d1485589-9260-4d31-90a8-7479d1a1713e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:09.942596 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d1485589-9260-4d31-90a8-7479d1a1713e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:09.943040 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d1485589-9260-4d31-90a8-7479d1a1713e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:10.083374 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d1485589-9260-4d31-90a8-7479d1a1713e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:10.092977 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d1485589-9260-4d31-90a8-7479d1a1713e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1265 microversion: 2.1 time: 0.193900 Jul 05 10:12:10.117941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 147/332] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:09 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1265 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:10.941673 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8ef0624b-e1d0-4b35-9f7b-deabeed85b16 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:10.944049 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8ef0624b-e1d0-4b35-9f7b-deabeed85b16 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:10.978890 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ef0624b-e1d0-4b35-9f7b-deabeed85b16 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:10.981323 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ef0624b-e1d0-4b35-9f7b-deabeed85b16 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:10.982838 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ef0624b-e1d0-4b35-9f7b-deabeed85b16 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:10.984089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ef0624b-e1d0-4b35-9f7b-deabeed85b16 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:11.110770 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-354a5a25-ae84-4d6f-920e-8388a42479a4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:11.111378 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-354a5a25-ae84-4d6f-920e-8388a42479a4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:11.114432 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=79,nova_api:UPDATE=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:11.122643 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-354a5a25-ae84-4d6f-920e-8388a42479a4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:11.123098 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-354a5a25-ae84-4d6f-920e-8388a42479a4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:11.123876 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-354a5a25-ae84-4d6f-920e-8388a42479a4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:11.124312 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-354a5a25-ae84-4d6f-920e-8388a42479a4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:11.211645 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:11.350552 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-354a5a25-ae84-4d6f-920e-8388a42479a4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:11.363676 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-354a5a25-ae84-4d6f-920e-8388a42479a4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1271 microversion: 2.1 time: 0.254374 Jul 05 10:12:11.365680 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 148/333] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:11 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1271 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:11.408128 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8ef0624b-e1d0-4b35-9f7b-deabeed85b16 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:11.412986 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=100 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:11.422838 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8ef0624b-e1d0-4b35-9f7b-deabeed85b16 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.494093 Jul 05 10:12:11.423514 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 186/334] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:10 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:12.386229 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f2fd13c2-6e67-45a8-a8dc-46e2a72450ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:12.388774 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f2fd13c2-6e67-45a8-a8dc-46e2a72450ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:12.401350 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f2fd13c2-6e67-45a8-a8dc-46e2a72450ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:12.404339 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f2fd13c2-6e67-45a8-a8dc-46e2a72450ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:12.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f2fd13c2-6e67-45a8-a8dc-46e2a72450ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:12.408425 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f2fd13c2-6e67-45a8-a8dc-46e2a72450ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:12.440324 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-43d51caf-721f-4f79-a547-af5e61e55758 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:12.441177 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-43d51caf-721f-4f79-a547-af5e61e55758 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:12.454113 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-43d51caf-721f-4f79-a547-af5e61e55758 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:12.455459 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-43d51caf-721f-4f79-a547-af5e61e55758 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:12.461237 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-43d51caf-721f-4f79-a547-af5e61e55758 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:12.463940 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-43d51caf-721f-4f79-a547-af5e61e55758 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:13.040255 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f2fd13c2-6e67-45a8-a8dc-46e2a72450ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:13.053877 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f2fd13c2-6e67-45a8-a8dc-46e2a72450ab tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1313 microversion: 2.1 time: 0.672303 Jul 05 10:12:13.057254 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 149/335] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:12 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1313 bytes in 679 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:13.076460 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-43d51caf-721f-4f79-a547-af5e61e55758 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:13.084284 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-43d51caf-721f-4f79-a547-af5e61e55758 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.647631 Jul 05 10:12:13.091770 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 187/336] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:12 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:14.072152 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-67030ebf-ce0e-4c2f-949f-eae84b5388f8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:14.074578 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-67030ebf-ce0e-4c2f-949f-eae84b5388f8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:14.084030 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-67030ebf-ce0e-4c2f-949f-eae84b5388f8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:14.085101 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-67030ebf-ce0e-4c2f-949f-eae84b5388f8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:14.085898 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-67030ebf-ce0e-4c2f-949f-eae84b5388f8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:14.086342 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-67030ebf-ce0e-4c2f-949f-eae84b5388f8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:14.109213 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-4bc3281c-03e1-4732-b195-3c6073fd00ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:14.110512 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-4bc3281c-03e1-4732-b195-3c6073fd00ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:14.123321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4bc3281c-03e1-4732-b195-3c6073fd00ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:14.129091 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4bc3281c-03e1-4732-b195-3c6073fd00ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:14.130563 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4bc3281c-03e1-4732-b195-3c6073fd00ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:14.131315 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4bc3281c-03e1-4732-b195-3c6073fd00ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:14.647940 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-67030ebf-ce0e-4c2f-949f-eae84b5388f8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:14.657961 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-67030ebf-ce0e-4c2f-949f-eae84b5388f8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1313 microversion: 2.1 time: 0.590509 Jul 05 10:12:14.672896 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 150/337] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:14 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1313 bytes in 606 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:14.716252 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4bc3281c-03e1-4732-b195-3c6073fd00ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:14.729338 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-4bc3281c-03e1-4732-b195-3c6073fd00ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1522 microversion: 2.1 time: 0.622990 Jul 05 10:12:14.730011 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 188/338] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:14 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1522 bytes in 626 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:14.740088 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-18531925-2fa4-4091-95fc-3408edb27200 req-b032344d-b495-415c-977a-614940606f71 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b6f3a8ca-f372-464b-b604-87af58bbd34c", "tag": "8459a921-58ac-4cef-9bd3-c506f7a8ff2a"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:12:14.751279 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-18531925-2fa4-4091-95fc-3408edb27200 req-b032344d-b495-415c-977a-614940606f71 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:14.752128 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-18531925-2fa4-4091-95fc-3408edb27200 req-b032344d-b495-415c-977a-614940606f71 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:14.765218 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-18531925-2fa4-4091-95fc-3408edb27200 req-b032344d-b495-415c-977a-614940606f71 service nova] Creating event network-changed:8459a921-58ac-4cef-9bd3-c506f7a8ff2a for instance b6f3a8ca-f372-464b-b604-87af58bbd34c on hv-cinder-maste Jul 05 10:12:14.773126 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-18531925-2fa4-4091-95fc-3408edb27200 req-b032344d-b495-415c-977a-614940606f71 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036729 Jul 05 10:12:14.773988 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 151/339] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:12:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:12:15.689194 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-524f8200-3005-44a7-b763-0a796db89ed1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:15.690509 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-524f8200-3005-44a7-b763-0a796db89ed1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:15.697474 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-524f8200-3005-44a7-b763-0a796db89ed1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:15.697830 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-524f8200-3005-44a7-b763-0a796db89ed1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:15.699004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-524f8200-3005-44a7-b763-0a796db89ed1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:15.699606 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-524f8200-3005-44a7-b763-0a796db89ed1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:15.748793 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-08fa4e6c-019c-43e4-af6a-ed8fbfe96724 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:15.749557 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-08fa4e6c-019c-43e4-af6a-ed8fbfe96724 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:15.770796 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-08fa4e6c-019c-43e4-af6a-ed8fbfe96724 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:15.771261 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-08fa4e6c-019c-43e4-af6a-ed8fbfe96724 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:15.772053 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-08fa4e6c-019c-43e4-af6a-ed8fbfe96724 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:15.772521 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-08fa4e6c-019c-43e4-af6a-ed8fbfe96724 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:16.007759 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-524f8200-3005-44a7-b763-0a796db89ed1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:16.008240 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-08fa4e6c-019c-43e4-af6a-ed8fbfe96724 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:16.029822 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-524f8200-3005-44a7-b763-0a796db89ed1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1313 microversion: 2.1 time: 0.344324 Jul 05 10:12:16.030698 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 189/340] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:15 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1313 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:16.033604 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-08fa4e6c-019c-43e4-af6a-ed8fbfe96724 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1502 microversion: 2.1 time: 0.290398 Jul 05 10:12:16.034171 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 152/341] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:15 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1502 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:16.059200 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9d841e5e-d0fe-4415-9e34-a85089d4292e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:16.059819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9d841e5e-d0fe-4415-9e34-a85089d4292e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:16.067913 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9d841e5e-d0fe-4415-9e34-a85089d4292e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:16.068652 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9d841e5e-d0fe-4415-9e34-a85089d4292e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:16.069542 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9d841e5e-d0fe-4415-9e34-a85089d4292e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:16.069891 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9d841e5e-d0fe-4415-9e34-a85089d4292e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:16.340835 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9d841e5e-d0fe-4415-9e34-a85089d4292e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:16.349220 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9d841e5e-d0fe-4415-9e34-a85089d4292e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1515 microversion: 2.1 time: 0.295698 Jul 05 10:12:16.351075 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 190/342] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:16 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1515 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:17.046477 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-732a8bdc-7d3a-4ab1-9017-2b5d6427f433 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:17.047031 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-732a8bdc-7d3a-4ab1-9017-2b5d6427f433 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:17.054149 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-732a8bdc-7d3a-4ab1-9017-2b5d6427f433 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:17.054927 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-732a8bdc-7d3a-4ab1-9017-2b5d6427f433 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:17.055746 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-732a8bdc-7d3a-4ab1-9017-2b5d6427f433 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:17.056338 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-732a8bdc-7d3a-4ab1-9017-2b5d6427f433 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:17.257428 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-732a8bdc-7d3a-4ab1-9017-2b5d6427f433 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:17.265509 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-732a8bdc-7d3a-4ab1-9017-2b5d6427f433 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1313 microversion: 2.1 time: 0.222960 Jul 05 10:12:17.266106 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 153/343] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:17 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1313 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:18.282833 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-253a0b78-5371-4531-9743-22aabb83eeb5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:18.283538 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-253a0b78-5371-4531-9743-22aabb83eeb5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:18.293483 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-253a0b78-5371-4531-9743-22aabb83eeb5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:18.293839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-253a0b78-5371-4531-9743-22aabb83eeb5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:18.294844 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-253a0b78-5371-4531-9743-22aabb83eeb5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:18.295479 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-253a0b78-5371-4531-9743-22aabb83eeb5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:18.524105 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-253a0b78-5371-4531-9743-22aabb83eeb5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:18.531530 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-253a0b78-5371-4531-9743-22aabb83eeb5 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1313 microversion: 2.1 time: 0.252751 Jul 05 10:12:18.534450 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 191/344] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:18 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1313 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:18.884320 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-cb1ed45b-b053-49aa-a5be-3034a86ec79a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:12:18.888281 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-cb1ed45b-b053-49aa-a5be-3034a86ec79a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:18.899765 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:18.909926 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cb1ed45b-b053-49aa-a5be-3034a86ec79a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:18.910375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cb1ed45b-b053-49aa-a5be-3034a86ec79a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:18.911141 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cb1ed45b-b053-49aa-a5be-3034a86ec79a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:18.911576 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cb1ed45b-b053-49aa-a5be-3034a86ec79a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:18.956573 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-cb1ed45b-b053-49aa-a5be-3034a86ec79a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lazy-loading 'flavor' on Instance uuid f6c29e5f-2d52-4c7a-9122-11dbaf998421 {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:12:19.315028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-cb1ed45b-b053-49aa-a5be-3034a86ec79a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "POST /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421/action" status: 200 len: 32 microversion: 2.1 time: 0.436528 Jul 05 10:12:19.315905 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 154/345] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:12:18 2022] POST /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421/action => generated 32 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 05 10:12:19.326222 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b018e0af-e094-4594-9bdd-7a38d949ed1c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:19.327658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b018e0af-e094-4594-9bdd-7a38d949ed1c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:19.331079 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:19.331793 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=6 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:19.335707 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b018e0af-e094-4594-9bdd-7a38d949ed1c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:19.336418 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b018e0af-e094-4594-9bdd-7a38d949ed1c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:19.337208 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b018e0af-e094-4594-9bdd-7a38d949ed1c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:19.337644 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b018e0af-e094-4594-9bdd-7a38d949ed1c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:19.499256 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b018e0af-e094-4594-9bdd-7a38d949ed1c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:19.507201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b018e0af-e094-4594-9bdd-7a38d949ed1c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.185903 Jul 05 10:12:19.507782 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 192/346] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:19 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:19.550892 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e7ff968f-baa9-4067-8481-97d5f989b6b8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:19.551609 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e7ff968f-baa9-4067-8481-97d5f989b6b8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:19.557956 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e7ff968f-baa9-4067-8481-97d5f989b6b8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:19.558405 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e7ff968f-baa9-4067-8481-97d5f989b6b8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:19.559767 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e7ff968f-baa9-4067-8481-97d5f989b6b8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:19.560354 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e7ff968f-baa9-4067-8481-97d5f989b6b8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:19.706085 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e7ff968f-baa9-4067-8481-97d5f989b6b8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:19.715216 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e7ff968f-baa9-4067-8481-97d5f989b6b8 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1313 microversion: 2.1 time: 0.168327 Jul 05 10:12:19.717684 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 155/347] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:19 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1313 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:20.524603 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-7cac2481-1df7-41de-9c9b-1ef964592a81 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:20.525212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7cac2481-1df7-41de-9c9b-1ef964592a81 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:20.531726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7cac2481-1df7-41de-9c9b-1ef964592a81 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:20.532353 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7cac2481-1df7-41de-9c9b-1ef964592a81 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:20.533294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7cac2481-1df7-41de-9c9b-1ef964592a81 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:20.533875 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7cac2481-1df7-41de-9c9b-1ef964592a81 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:20.720268 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-7cac2481-1df7-41de-9c9b-1ef964592a81 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:20.735945 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5df1c137-fbfe-48b7-b4bd-f9dfb00bed69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:20.736650 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5df1c137-fbfe-48b7-b4bd-f9dfb00bed69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:20.742414 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-7cac2481-1df7-41de-9c9b-1ef964592a81 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.221687 Jul 05 10:12:20.755770 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5df1c137-fbfe-48b7-b4bd-f9dfb00bed69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:20.756220 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5df1c137-fbfe-48b7-b4bd-f9dfb00bed69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:20.756728 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 193/348] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:20 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:20.757606 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5df1c137-fbfe-48b7-b4bd-f9dfb00bed69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:20.758052 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5df1c137-fbfe-48b7-b4bd-f9dfb00bed69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:20.931432 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5df1c137-fbfe-48b7-b4bd-f9dfb00bed69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:20.940487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5df1c137-fbfe-48b7-b4bd-f9dfb00bed69 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1313 microversion: 2.1 time: 0.210897 Jul 05 10:12:20.950022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 156/349] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:20 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1313 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:21.759763 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3cf94b4c-d36b-46e7-90c0-768226017ef0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:21.760463 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3cf94b4c-d36b-46e7-90c0-768226017ef0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:21.771866 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3cf94b4c-d36b-46e7-90c0-768226017ef0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:21.772292 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3cf94b4c-d36b-46e7-90c0-768226017ef0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:21.775138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3cf94b4c-d36b-46e7-90c0-768226017ef0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:21.775584 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3cf94b4c-d36b-46e7-90c0-768226017ef0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:21.960580 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7473bb1f-e6c0-40a5-b4ba-a077fe5f0fc3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:21.962863 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7473bb1f-e6c0-40a5-b4ba-a077fe5f0fc3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:21.974296 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7473bb1f-e6c0-40a5-b4ba-a077fe5f0fc3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:21.975116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7473bb1f-e6c0-40a5-b4ba-a077fe5f0fc3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:21.976799 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7473bb1f-e6c0-40a5-b4ba-a077fe5f0fc3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:21.987431 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7473bb1f-e6c0-40a5-b4ba-a077fe5f0fc3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:22.158762 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-3cf94b4c-d36b-46e7-90c0-768226017ef0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:22.172518 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3cf94b4c-d36b-46e7-90c0-768226017ef0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.413545 Jul 05 10:12:22.173131 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 194/350] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:21 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:22.275027 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-7473bb1f-e6c0-40a5-b4ba-a077fe5f0fc3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:22.287654 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7473bb1f-e6c0-40a5-b4ba-a077fe5f0fc3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1313 microversion: 2.1 time: 0.333982 Jul 05 10:12:22.290386 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 157/351] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:21 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1313 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:23.190442 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3538b2c3-d9e3-4a69-8adb-a5c989344120 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:23.191131 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3538b2c3-d9e3-4a69-8adb-a5c989344120 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:23.198954 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3538b2c3-d9e3-4a69-8adb-a5c989344120 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:23.199548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3538b2c3-d9e3-4a69-8adb-a5c989344120 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:23.201065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3538b2c3-d9e3-4a69-8adb-a5c989344120 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:23.201511 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3538b2c3-d9e3-4a69-8adb-a5c989344120 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:23.306080 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-84be159b-5a2b-4313-9446-a8495e6a012c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:23.306961 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-84be159b-5a2b-4313-9446-a8495e6a012c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:23.316496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-84be159b-5a2b-4313-9446-a8495e6a012c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:23.316496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-84be159b-5a2b-4313-9446-a8495e6a012c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:23.316496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-84be159b-5a2b-4313-9446-a8495e6a012c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:23.316899 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-84be159b-5a2b-4313-9446-a8495e6a012c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:23.379948 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-3538b2c3-d9e3-4a69-8adb-a5c989344120 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:23.389015 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3538b2c3-d9e3-4a69-8adb-a5c989344120 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.204081 Jul 05 10:12:23.389821 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 195/352] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:23 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:23.482007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-84be159b-5a2b-4313-9446-a8495e6a012c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:23.490722 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-84be159b-5a2b-4313-9446-a8495e6a012c tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1313 microversion: 2.1 time: 0.189095 Jul 05 10:12:23.491315 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 158/353] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:23 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1313 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:23.716654 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-e189a653-2ea0-4fc1-a029-9044123a13c8 req-b062feaf-9174-4ea4-82c1-c118f86a6ad3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b6f3a8ca-f372-464b-b604-87af58bbd34c", "name": "network-vif-plugged", "status": "completed", "tag": "8459a921-58ac-4cef-9bd3-c506f7a8ff2a"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:12:23.728213 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-e189a653-2ea0-4fc1-a029-9044123a13c8 req-b062feaf-9174-4ea4-82c1-c118f86a6ad3 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:23.728925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-e189a653-2ea0-4fc1-a029-9044123a13c8 req-b062feaf-9174-4ea4-82c1-c118f86a6ad3 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:23.742099 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-e189a653-2ea0-4fc1-a029-9044123a13c8 req-b062feaf-9174-4ea4-82c1-c118f86a6ad3 service nova] Creating event network-vif-plugged:8459a921-58ac-4cef-9bd3-c506f7a8ff2a for instance b6f3a8ca-f372-464b-b604-87af58bbd34c on hv-cinder-maste Jul 05 10:12:23.749035 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-e189a653-2ea0-4fc1-a029-9044123a13c8 req-b062feaf-9174-4ea4-82c1-c118f86a6ad3 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036261 Jul 05 10:12:23.749973 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 196/354] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:12:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:12:24.406523 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-58dd77d5-9d05-4638-942f-2c55925a0abd tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:24.407141 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-58dd77d5-9d05-4638-942f-2c55925a0abd tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:24.416049 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-58dd77d5-9d05-4638-942f-2c55925a0abd tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:24.416989 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-58dd77d5-9d05-4638-942f-2c55925a0abd tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:24.417925 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-58dd77d5-9d05-4638-942f-2c55925a0abd tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:24.419300 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-58dd77d5-9d05-4638-942f-2c55925a0abd tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:24.508226 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-926de803-e1fb-459a-9312-9c3d6cc369b3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:24.509032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-926de803-e1fb-459a-9312-9c3d6cc369b3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:24.518427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-926de803-e1fb-459a-9312-9c3d6cc369b3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:24.521101 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-926de803-e1fb-459a-9312-9c3d6cc369b3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:24.523285 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-926de803-e1fb-459a-9312-9c3d6cc369b3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:24.523795 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-926de803-e1fb-459a-9312-9c3d6cc369b3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:24.664640 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-58dd77d5-9d05-4638-942f-2c55925a0abd tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:24.678646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-58dd77d5-9d05-4638-942f-2c55925a0abd tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.276336 Jul 05 10:12:24.679526 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 159/355] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:24 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:24.743088 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-926de803-e1fb-459a-9312-9c3d6cc369b3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:24.750563 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-926de803-e1fb-459a-9312-9c3d6cc369b3 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1313 microversion: 2.1 time: 0.246526 Jul 05 10:12:24.751118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 197/356] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:24 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1313 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:25.699819 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e83ead65-1144-40ff-ab1a-7548eac7ac32 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:25.700860 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e83ead65-1144-40ff-ab1a-7548eac7ac32 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:25.708172 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e83ead65-1144-40ff-ab1a-7548eac7ac32 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:25.708668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e83ead65-1144-40ff-ab1a-7548eac7ac32 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:25.709440 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e83ead65-1144-40ff-ab1a-7548eac7ac32 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:25.709873 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e83ead65-1144-40ff-ab1a-7548eac7ac32 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:25.767136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-4f77760f-d6a2-4e9e-b565-ff59c873f5d4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:25.768259 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-4f77760f-d6a2-4e9e-b565-ff59c873f5d4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:25.778584 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f77760f-d6a2-4e9e-b565-ff59c873f5d4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:25.781043 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f77760f-d6a2-4e9e-b565-ff59c873f5d4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:25.781846 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f77760f-d6a2-4e9e-b565-ff59c873f5d4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:25.782288 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f77760f-d6a2-4e9e-b565-ff59c873f5d4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:25.949983 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e83ead65-1144-40ff-ab1a-7548eac7ac32 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:25.959394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e83ead65-1144-40ff-ab1a-7548eac7ac32 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.267550 Jul 05 10:12:25.960021 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 160/357] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:25 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:25.984085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4f77760f-d6a2-4e9e-b565-ff59c873f5d4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:25.991721 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-4f77760f-d6a2-4e9e-b565-ff59c873f5d4 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1313 microversion: 2.1 time: 0.228644 Jul 05 10:12:25.992289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 198/358] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:25 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1313 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:26.977384 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-dbe3a562-c3b0-4317-b8c9-85325f922d15 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:26.978727 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-dbe3a562-c3b0-4317-b8c9-85325f922d15 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:26.987035 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dbe3a562-c3b0-4317-b8c9-85325f922d15 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:26.987667 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dbe3a562-c3b0-4317-b8c9-85325f922d15 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:26.988459 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dbe3a562-c3b0-4317-b8c9-85325f922d15 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:26.989169 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dbe3a562-c3b0-4317-b8c9-85325f922d15 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:27.008820 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-60b8999e-fee0-43f4-b8f4-76a2a76d3bbb tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:27.010454 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-60b8999e-fee0-43f4-b8f4-76a2a76d3bbb tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:27.019876 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-60b8999e-fee0-43f4-b8f4-76a2a76d3bbb tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:27.021889 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-60b8999e-fee0-43f4-b8f4-76a2a76d3bbb tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:27.023715 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-60b8999e-fee0-43f4-b8f4-76a2a76d3bbb tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:27.024822 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-60b8999e-fee0-43f4-b8f4-76a2a76d3bbb tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:27.345050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-dbe3a562-c3b0-4317-b8c9-85325f922d15 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:27.353606 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-dbe3a562-c3b0-4317-b8c9-85325f922d15 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.380205 Jul 05 10:12:27.362894 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 161/359] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:26 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:27.405609 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-60b8999e-fee0-43f4-b8f4-76a2a76d3bbb tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:27.414899 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-60b8999e-fee0-43f4-b8f4-76a2a76d3bbb tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1497 microversion: 2.1 time: 0.410272 Jul 05 10:12:27.415933 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 199/360] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:27 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1497 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:28.218396 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-aabf6567-070d-4a29-b7e8-962a7ea76a0e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:28.218957 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-aabf6567-070d-4a29-b7e8-962a7ea76a0e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:28.225271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-aabf6567-070d-4a29-b7e8-962a7ea76a0e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:28.225800 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-aabf6567-070d-4a29-b7e8-962a7ea76a0e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:28.226654 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-aabf6567-070d-4a29-b7e8-962a7ea76a0e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:28.227159 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-aabf6567-070d-4a29-b7e8-962a7ea76a0e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:28.253295 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-aabf6567-070d-4a29-b7e8-962a7ea76a0e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Going to try to terminate instance {{(pid=117812) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:12:28.281882 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:28.354714 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-aabf6567-070d-4a29-b7e8-962a7ea76a0e tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 204 len: 0 microversion: 2.1 time: 0.139844 Jul 05 10:12:28.355752 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 162/361] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:12:28 2022] DELETE /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:12:28.368499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e325c36f-7063-43f2-8b21-a8165f8062f1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:28.369124 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e325c36f-7063-43f2-8b21-a8165f8062f1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:28.375701 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e325c36f-7063-43f2-8b21-a8165f8062f1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:28.376152 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e325c36f-7063-43f2-8b21-a8165f8062f1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:28.379513 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e325c36f-7063-43f2-8b21-a8165f8062f1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:28.379985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e325c36f-7063-43f2-8b21-a8165f8062f1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:28.382753 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-34633df4-3563-4f75-9816-9787e6b56fc2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:28.385916 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-34633df4-3563-4f75-9816-9787e6b56fc2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:28.394209 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-34633df4-3563-4f75-9816-9787e6b56fc2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:28.394670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-34633df4-3563-4f75-9816-9787e6b56fc2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:28.395507 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-34633df4-3563-4f75-9816-9787e6b56fc2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:28.396124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-34633df4-3563-4f75-9816-9787e6b56fc2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:28.910046 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e325c36f-7063-43f2-8b21-a8165f8062f1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:28.918271 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e325c36f-7063-43f2-8b21-a8165f8062f1 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1503 microversion: 2.1 time: 0.554024 Jul 05 10:12:28.918876 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 200/362] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:28 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1503 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:28.933490 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-34633df4-3563-4f75-9816-9787e6b56fc2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:28.942304 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-34633df4-3563-4f75-9816-9787e6b56fc2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.566022 Jul 05 10:12:28.943199 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 163/363] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:28 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:29.946198 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d207c806-d6ab-4760-9ce7-4137c57ec380 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:29.947884 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d207c806-d6ab-4760-9ce7-4137c57ec380 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:29.957511 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d207c806-d6ab-4760-9ce7-4137c57ec380 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:29.958190 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d207c806-d6ab-4760-9ce7-4137c57ec380 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:29.959265 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d207c806-d6ab-4760-9ce7-4137c57ec380 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:29.959858 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d207c806-d6ab-4760-9ce7-4137c57ec380 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:29.961517 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-13981f11-dcfc-47ca-9187-b5cc77c2bb26 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:29.962174 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-13981f11-dcfc-47ca-9187-b5cc77c2bb26 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:29.974700 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-13981f11-dcfc-47ca-9187-b5cc77c2bb26 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:29.976193 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-13981f11-dcfc-47ca-9187-b5cc77c2bb26 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:29.977160 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-13981f11-dcfc-47ca-9187-b5cc77c2bb26 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:29.977850 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-13981f11-dcfc-47ca-9187-b5cc77c2bb26 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:30.326562 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d207c806-d6ab-4760-9ce7-4137c57ec380 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:30.338031 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d207c806-d6ab-4760-9ce7-4137c57ec380 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1503 microversion: 2.1 time: 0.402110 Jul 05 10:12:30.339968 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 201/364] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:29 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1503 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:30.467674 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-13981f11-dcfc-47ca-9187-b5cc77c2bb26 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:30.485806 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-13981f11-dcfc-47ca-9187-b5cc77c2bb26 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.529080 Jul 05 10:12:30.486909 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 164/365] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:29 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:31.169642 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-374bbc96-5fbb-450f-9cf4-c95e2f8dcc0a req-0d717d9d-b0a6-4dc7-a494-d57230318d87 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b6f3a8ca-f372-464b-b604-87af58bbd34c", "name": "network-vif-deleted", "tag": "8459a921-58ac-4cef-9bd3-c506f7a8ff2a"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:12:31.183355 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-374bbc96-5fbb-450f-9cf4-c95e2f8dcc0a req-0d717d9d-b0a6-4dc7-a494-d57230318d87 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:31.185813 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-374bbc96-5fbb-450f-9cf4-c95e2f8dcc0a req-0d717d9d-b0a6-4dc7-a494-d57230318d87 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:31.208141 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-374bbc96-5fbb-450f-9cf4-c95e2f8dcc0a req-0d717d9d-b0a6-4dc7-a494-d57230318d87 service nova] Creating event network-vif-deleted:8459a921-58ac-4cef-9bd3-c506f7a8ff2a for instance b6f3a8ca-f372-464b-b604-87af58bbd34c on hv-cinder-maste Jul 05 10:12:31.215230 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-374bbc96-5fbb-450f-9cf4-c95e2f8dcc0a req-0d717d9d-b0a6-4dc7-a494-d57230318d87 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050200 Jul 05 10:12:31.217610 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 202/366] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:12:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:12:31.358568 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-54368c59-e3c1-40b5-ba65-aeefb8e72f59 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:31.362298 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-54368c59-e3c1-40b5-ba65-aeefb8e72f59 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:31.390590 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-54368c59-e3c1-40b5-ba65-aeefb8e72f59 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:31.391063 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-54368c59-e3c1-40b5-ba65-aeefb8e72f59 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:31.392143 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-54368c59-e3c1-40b5-ba65-aeefb8e72f59 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:31.392621 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-54368c59-e3c1-40b5-ba65-aeefb8e72f59 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:31.503489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d452c930-72ae-4c74-80b0-69f9e42f7845 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:31.504190 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d452c930-72ae-4c74-80b0-69f9e42f7845 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:31.516791 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d452c930-72ae-4c74-80b0-69f9e42f7845 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:31.517243 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d452c930-72ae-4c74-80b0-69f9e42f7845 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:31.519439 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d452c930-72ae-4c74-80b0-69f9e42f7845 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:31.525042 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d452c930-72ae-4c74-80b0-69f9e42f7845 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:31.573766 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-54368c59-e3c1-40b5-ba65-aeefb8e72f59 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2d25be9c4674aa09b2975f1c6bc8c52', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c48f677354594fdc839367fff8b3dc29', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:31.590611 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-54368c59-e3c1-40b5-ba65-aeefb8e72f59 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 200 len: 1298 microversion: 2.1 time: 0.234757 Jul 05 10:12:31.592775 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 165/367] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:31 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 1298 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:31.729450 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d452c930-72ae-4c74-80b0-69f9e42f7845 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:31.739473 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d452c930-72ae-4c74-80b0-69f9e42f7845 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.239845 Jul 05 10:12:31.740766 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 203/368] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:31 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:32.613230 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f621d509-0696-41cd-8525-8bfebb0abb74 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:32.614150 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f621d509-0696-41cd-8525-8bfebb0abb74 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] [instance: b6f3a8ca-f372-464b-b604-87af58bbd34c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:32.624439 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f621d509-0696-41cd-8525-8bfebb0abb74 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:32.625822 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f621d509-0696-41cd-8525-8bfebb0abb74 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:32.626991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f621d509-0696-41cd-8525-8bfebb0abb74 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:32.627581 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f621d509-0696-41cd-8525-8bfebb0abb74 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:32.666264 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-f621d509-0696-41cd-8525-8bfebb0abb74 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] HTTP exception thrown: Instance b6f3a8ca-f372-464b-b604-87af58bbd34c could not be found. Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f621d509-0696-41cd-8525-8bfebb0abb74 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Request method failure captured: Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c HTTP/1.1 Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA4nwoqtNHVEvvZjhlsgccGBfHw_4GX7-bJJaYYcYiHLm2podokcNX5Ngzj_K06wB_xDRq09afqP_YQKU63PBkD-_XweEXGvhD1Dj4YQljtZfjnDCsFoKAy3jWCECVjLpwSdQjvsbXx0wRW_ol9Kz--_dhmWVjih19FrCu8OSrTb8hA Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: c48f677354594fdc839367fff8b3dc29 Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-VolumesActionsTest-28417478 Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:12:32.668109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:12:32.670045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/c48f677354594fdc839367fff8b3dc29"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_c48f677354594fdc839367fff8b3dc29"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/c48f677354594fdc839367fff8b3dc29"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/c48f677354594fdc839367fff8b3dc29"}]}] Jul 05 10:12:32.670045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-VolumesActionsTest-28417478 Jul 05 10:12:32.670045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: c48f677354594fdc839367fff8b3dc29 Jul 05 10:12:32.670045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-VolumesActionsTest-28417478 Jul 05 10:12:32.670045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-VolumesActionsTest-28417478-project Jul 05 10:12:32.670045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:12:32.670045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:12:32.670045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: d2d25be9c4674aa09b2975f1c6bc8c52 Jul 05 10:12:32.670045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-VolumesActionsTest-28417478-project Jul 05 10:12:32.670045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:12:32.670045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance b6f3a8ca-f372-464b-b604-87af58bbd34c could not be found. Jul 05 10:12:32.670045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': 'b6f3a8ca-f372-464b-b604-87af58bbd34c'} Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b6f3a8ca-f372-464b-b604-87af58bbd34c could not be found. Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:12:32.670738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b6f3a8ca-f372-464b-b604-87af58bbd34c could not be found. Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b6f3a8ca-f372-464b-b604-87af58bbd34c could not be found. Jul 05 10:12:32.671933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:32.674565 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f621d509-0696-41cd-8525-8bfebb0abb74 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] Returning 404 to user: Instance b6f3a8ca-f372-464b-b604-87af58bbd34c could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:12:32.675602 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f621d509-0696-41cd-8525-8bfebb0abb74 tempest-VolumesActionsTest-28417478 tempest-VolumesActionsTest-28417478-project] 10.222.0.20 "GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c" status: 404 len: 111 microversion: 2.1 time: 0.069249 Jul 05 10:12:32.676709 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 166/369] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:32 2022] GET /compute/v2.1/servers/b6f3a8ca-f372-464b-b604-87af58bbd34c => generated 111 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:12:32.758538 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2ff5c25b-48bf-43a6-a076-3f41f0ec20c8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:32.759208 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2ff5c25b-48bf-43a6-a076-3f41f0ec20c8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:32.770566 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2ff5c25b-48bf-43a6-a076-3f41f0ec20c8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:32.771480 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2ff5c25b-48bf-43a6-a076-3f41f0ec20c8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:32.772544 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2ff5c25b-48bf-43a6-a076-3f41f0ec20c8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:32.774862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2ff5c25b-48bf-43a6-a076-3f41f0ec20c8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:33.128974 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2ff5c25b-48bf-43a6-a076-3f41f0ec20c8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:33.133558 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=100 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:33.145930 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2ff5c25b-48bf-43a6-a076-3f41f0ec20c8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.390700 Jul 05 10:12:33.148651 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 204/370] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:32 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:34.164660 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5ddbb3cb-5375-45d8-9c9e-05a978b4737c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:34.165258 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5ddbb3cb-5375-45d8-9c9e-05a978b4737c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:34.174925 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5ddbb3cb-5375-45d8-9c9e-05a978b4737c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:34.175382 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5ddbb3cb-5375-45d8-9c9e-05a978b4737c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:34.176172 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5ddbb3cb-5375-45d8-9c9e-05a978b4737c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:34.176639 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5ddbb3cb-5375-45d8-9c9e-05a978b4737c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:34.515525 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5ddbb3cb-5375-45d8-9c9e-05a978b4737c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:34.531180 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5ddbb3cb-5375-45d8-9c9e-05a978b4737c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.365829 Jul 05 10:12:34.541806 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 167/371] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:34 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:35.549124 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-58dcf4dd-85f1-4f3b-ad3f-e722d1e2600b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:35.549577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-58dcf4dd-85f1-4f3b-ad3f-e722d1e2600b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:35.557291 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-58dcf4dd-85f1-4f3b-ad3f-e722d1e2600b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:35.558335 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-58dcf4dd-85f1-4f3b-ad3f-e722d1e2600b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:35.559960 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-58dcf4dd-85f1-4f3b-ad3f-e722d1e2600b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:35.561543 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-58dcf4dd-85f1-4f3b-ad3f-e722d1e2600b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:35.733426 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-58dcf4dd-85f1-4f3b-ad3f-e722d1e2600b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:35.751817 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-58dcf4dd-85f1-4f3b-ad3f-e722d1e2600b tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.205132 Jul 05 10:12:35.756534 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 205/372] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:35 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:36.773005 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3f1560d5-ecb9-462d-9c64-cfa7fd3b78c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:36.773635 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3f1560d5-ecb9-462d-9c64-cfa7fd3b78c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:36.781019 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3f1560d5-ecb9-462d-9c64-cfa7fd3b78c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:36.781476 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3f1560d5-ecb9-462d-9c64-cfa7fd3b78c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:36.782286 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3f1560d5-ecb9-462d-9c64-cfa7fd3b78c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:36.782919 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3f1560d5-ecb9-462d-9c64-cfa7fd3b78c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:36.944789 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-3f1560d5-ecb9-462d-9c64-cfa7fd3b78c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:36.955049 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3f1560d5-ecb9-462d-9c64-cfa7fd3b78c3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.185801 Jul 05 10:12:36.956253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 168/373] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:36 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:37.973085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5854db54-6230-4e50-aaad-5a55cf1907bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:37.973694 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5854db54-6230-4e50-aaad-5a55cf1907bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:37.979965 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5854db54-6230-4e50-aaad-5a55cf1907bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:37.980716 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5854db54-6230-4e50-aaad-5a55cf1907bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:37.981752 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5854db54-6230-4e50-aaad-5a55cf1907bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:37.982398 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5854db54-6230-4e50-aaad-5a55cf1907bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:38.265036 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5854db54-6230-4e50-aaad-5a55cf1907bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:38.281275 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5854db54-6230-4e50-aaad-5a55cf1907bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.311952 Jul 05 10:12:38.282068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 206/374] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:37 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:39.298747 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d7f11dd6-fa97-4605-bb28-c319db828f95 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:39.299608 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d7f11dd6-fa97-4605-bb28-c319db828f95 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:39.306613 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d7f11dd6-fa97-4605-bb28-c319db828f95 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:39.307136 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d7f11dd6-fa97-4605-bb28-c319db828f95 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:39.308138 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d7f11dd6-fa97-4605-bb28-c319db828f95 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:39.308676 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d7f11dd6-fa97-4605-bb28-c319db828f95 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:39.557371 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d7f11dd6-fa97-4605-bb28-c319db828f95 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:39.567190 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d7f11dd6-fa97-4605-bb28-c319db828f95 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.272291 Jul 05 10:12:39.568023 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 169/375] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:39 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:40.585018 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a4f6dc95-9394-4e3a-94d0-16c1471fc605 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:40.586339 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a4f6dc95-9394-4e3a-94d0-16c1471fc605 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:40.598614 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=44 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:40.600189 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a4f6dc95-9394-4e3a-94d0-16c1471fc605 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:40.600878 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a4f6dc95-9394-4e3a-94d0-16c1471fc605 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:40.601820 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a4f6dc95-9394-4e3a-94d0-16c1471fc605 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:40.602254 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a4f6dc95-9394-4e3a-94d0-16c1471fc605 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:40.771453 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a4f6dc95-9394-4e3a-94d0-16c1471fc605 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:40.781165 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a4f6dc95-9394-4e3a-94d0-16c1471fc605 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.200438 Jul 05 10:12:40.791343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 207/376] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:40 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:41.802717 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-883432c2-3f38-47b2-834d-f4b0403da219 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:41.804208 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-883432c2-3f38-47b2-834d-f4b0403da219 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:41.810986 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:41.816824 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-883432c2-3f38-47b2-834d-f4b0403da219 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:41.817284 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-883432c2-3f38-47b2-834d-f4b0403da219 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:41.818458 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-883432c2-3f38-47b2-834d-f4b0403da219 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:41.818706 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-883432c2-3f38-47b2-834d-f4b0403da219 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:42.038742 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-883432c2-3f38-47b2-834d-f4b0403da219 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:42.046582 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-883432c2-3f38-47b2-834d-f4b0403da219 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.250598 Jul 05 10:12:42.047761 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 170/377] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:41 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:42.364317 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0d349750-9177-42c7-9841-a6039b9215a0 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Action: 'create', calling method: .version_select at 0x7feb8a0449d0>, body: {"keypair": {"name": "tempest-TestShelveInstance-309513784", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHOFRHvnm1Ww+2RopjCu8A7lcQ3Fofk5bNNyapAVqyLVl9gRVYmWWXMwE012I8p6DvJiNO3g12qVrGZvkYkYvHYIJpJUv9HBL4tPyU3teEWilSMl7GqRDN9P/cWhhyjWmg=="}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:12:42.385732 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-0d349750-9177-42c7-9841-a6039b9215a0 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Getting quotas for project 9edab973767545988e178ddff76557ac. Resources: {'key_pairs'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:12:42.390548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-0d349750-9177-42c7-9841-a6039b9215a0 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Getting quotas for user 7e322227b42b4aaa99674afd906a706a and project 9edab973767545988e178ddff76557ac. Resources: {'key_pairs'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:12:42.428991 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0d349750-9177-42c7-9841-a6039b9215a0 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.241434 Jul 05 10:12:42.429862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 208/378] 10.222.0.20 () {60 vars in 1158 bytes} [Tue Jul 5 10:12:42 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:12:43.065617 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f9ecf5ef-6017-45d4-9342-64ed067700b7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:43.067029 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f9ecf5ef-6017-45d4-9342-64ed067700b7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:43.078658 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f9ecf5ef-6017-45d4-9342-64ed067700b7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:43.079125 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f9ecf5ef-6017-45d4-9342-64ed067700b7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:43.079919 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f9ecf5ef-6017-45d4-9342-64ed067700b7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:43.080364 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f9ecf5ef-6017-45d4-9342-64ed067700b7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:43.259110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f9ecf5ef-6017-45d4-9342-64ed067700b7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:43.271327 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f9ecf5ef-6017-45d4-9342-64ed067700b7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.211950 Jul 05 10:12:43.273813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 171/379] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:43 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:44.289707 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-051dc4a9-9e8f-4336-a649-0a8faf751962 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:44.290310 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-051dc4a9-9e8f-4336-a649-0a8faf751962 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:44.296429 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-051dc4a9-9e8f-4336-a649-0a8faf751962 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:44.296877 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-051dc4a9-9e8f-4336-a649-0a8faf751962 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:44.297723 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-051dc4a9-9e8f-4336-a649-0a8faf751962 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:44.298160 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-051dc4a9-9e8f-4336-a649-0a8faf751962 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:44.435544 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-051dc4a9-9e8f-4336-a649-0a8faf751962 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:44.444222 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-051dc4a9-9e8f-4336-a649-0a8faf751962 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.158413 Jul 05 10:12:44.444868 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 209/380] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:44 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:45.463214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-16ee32ec-1ac1-4f0e-8305-559bb383ecf1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:45.464731 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-16ee32ec-1ac1-4f0e-8305-559bb383ecf1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:45.476798 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-16ee32ec-1ac1-4f0e-8305-559bb383ecf1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:45.477477 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-16ee32ec-1ac1-4f0e-8305-559bb383ecf1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:45.478322 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-16ee32ec-1ac1-4f0e-8305-559bb383ecf1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:45.480268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-16ee32ec-1ac1-4f0e-8305-559bb383ecf1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:45.646690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-16ee32ec-1ac1-4f0e-8305-559bb383ecf1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:45.658535 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-16ee32ec-1ac1-4f0e-8305-559bb383ecf1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.200285 Jul 05 10:12:45.668693 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 172/381] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:45 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:46.488032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1462998651", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:12:46.675362 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-72937c27-a5de-4b83-b340-23e2bfd462c0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:46.675968 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-72937c27-a5de-4b83-b340-23e2bfd462c0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:46.682644 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72937c27-a5de-4b83-b340-23e2bfd462c0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:46.683088 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72937c27-a5de-4b83-b340-23e2bfd462c0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:46.683850 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72937c27-a5de-4b83-b340-23e2bfd462c0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:46.684287 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72937c27-a5de-4b83-b340-23e2bfd462c0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:46.823718 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-72937c27-a5de-4b83-b340-23e2bfd462c0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:46.839539 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-72937c27-a5de-4b83-b340-23e2bfd462c0 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.167907 Jul 05 10:12:46.840708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 173/382] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:46 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:47.857541 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e6f972e7-df16-4603-b77b-6201dba17889 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:47.858280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e6f972e7-df16-4603-b77b-6201dba17889 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:47.866274 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e6f972e7-df16-4603-b77b-6201dba17889 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:47.866726 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e6f972e7-df16-4603-b77b-6201dba17889 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:47.867503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e6f972e7-df16-4603-b77b-6201dba17889 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:47.867946 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e6f972e7-df16-4603-b77b-6201dba17889 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:48.003938 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e6f972e7-df16-4603-b77b-6201dba17889 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:48.011687 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e6f972e7-df16-4603-b77b-6201dba17889 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1501 microversion: 2.1 time: 0.158622 Jul 05 10:12:48.012564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 174/383] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:47 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1501 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:48.028187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d8a3dd87-8769-4de0-93ce-79d13a28bbaa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a5b505b2-bcbf-419c-8935-4cb62d692d06"}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:12:48.029422 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d8a3dd87-8769-4de0-93ce-79d13a28bbaa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:48.035445 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d8a3dd87-8769-4de0-93ce-79d13a28bbaa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:48.035901 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d8a3dd87-8769-4de0-93ce-79d13a28bbaa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:48.036704 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d8a3dd87-8769-4de0-93ce-79d13a28bbaa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:48.037144 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d8a3dd87-8769-4de0-93ce-79d13a28bbaa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:48.065720 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-d8a3dd87-8769-4de0-93ce-79d13a28bbaa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] HTTP exception thrown: Cannot 'attach_volume' instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 while it is in vm_state rescued Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d8a3dd87-8769-4de0-93ce-79d13a28bbaa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Request method failure captured: Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: POST /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421/os-volume_attachments HTTP/1.1 Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 74 Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA4LMwEWQwEVIGHW57tb-mCZDddObx4iwQyZ5gk_llO51wkTkuVjOLNqddatV2FzjtkzWxyWr7Rmd5kPAnAKyMQJw43XAdu8pkSZT9OsjCAGydI1FCbBCPFs_pXG35EW9p3Z8AlqJqZdFJIfQwAG269sW3HEsdORJuVOabGGWKLHWOI Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: 3f2199fa0bc54901aa24195a250a9e25 Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:12:48.067563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:12:48.068941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/3f2199fa0bc54901aa24195a250a9e25"}]}] Jul 05 10:12:48.068941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:12:48.068941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: 3f2199fa0bc54901aa24195a250a9e25 Jul 05 10:12:48.068941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:12:48.068941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-ServerRescueNegativeTestJSON-1125341258-project Jul 05 10:12:48.068941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:12:48.068941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:12:48.068941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: 248946c73cb245f9b8e6fe73e46133cf Jul 05 10:12:48.068941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1125341258-project Jul 05 10:12:48.068941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Jul 05 10:12:48.068941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {"volumeAttachment": {"volumeId": "a5b505b2-bcbf-419c-8935-4cb62d692d06"}} Jul 05 10:12:48.068941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Cannot 'attach_volume' instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 while it is in vm_state rescued Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'server_id': 'f6c29e5f-2d52-4c7a-9122-11dbaf998421', 'body': {'volumeAttachment': {'volumeId': 'a5b505b2-bcbf-419c-8935-4cb62d692d06'}}} Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 in vm_state rescued. Cannot attach_volume while the instance is in this state. Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:12:48.069803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 while it is in vm_state rescued Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:12:48.071124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:12:48.072439 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:12:48.072439 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:12:48.072439 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 while it is in vm_state rescued Jul 05 10:12:48.072439 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:12:48.072439 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d8a3dd87-8769-4de0-93ce-79d13a28bbaa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Returning 409 to user: Cannot 'attach_volume' instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 while it is in vm_state rescued {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:12:48.073560 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d8a3dd87-8769-4de0-93ce-79d13a28bbaa tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "POST /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.049423 Jul 05 10:12:48.074375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 175/384] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:12:48 2022] POST /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421/os-volume_attachments => generated 152 bytes in 51 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Jul 05 10:12:48.083202 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-9de68509-78f6-40c7-881c-7002577f8dc6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:12:48.083739 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-9de68509-78f6-40c7-881c-7002577f8dc6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:48.090421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9de68509-78f6-40c7-881c-7002577f8dc6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:48.090889 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9de68509-78f6-40c7-881c-7002577f8dc6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:48.091688 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9de68509-78f6-40c7-881c-7002577f8dc6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:48.092172 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9de68509-78f6-40c7-881c-7002577f8dc6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:48.163921 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-9de68509-78f6-40c7-881c-7002577f8dc6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "POST /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421/action" status: 202 len: 0 microversion: 2.1 time: 0.084521 Jul 05 10:12:48.164822 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 176/385] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:12:48 2022] POST /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421/action => generated 0 bytes in 86 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 05 10:12:48.175121 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-325c58de-79e5-4de2-a3b1-e6ad2c83ceba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:48.175966 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-325c58de-79e5-4de2-a3b1-e6ad2c83ceba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:48.183121 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-325c58de-79e5-4de2-a3b1-e6ad2c83ceba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:48.183601 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-325c58de-79e5-4de2-a3b1-e6ad2c83ceba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:48.184563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-325c58de-79e5-4de2-a3b1-e6ad2c83ceba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:48.184961 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-325c58de-79e5-4de2-a3b1-e6ad2c83ceba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:48.411787 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-325c58de-79e5-4de2-a3b1-e6ad2c83ceba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:48.419949 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-325c58de-79e5-4de2-a3b1-e6ad2c83ceba tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1509 microversion: 2.1 time: 0.250215 Jul 05 10:12:48.420723 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 177/386] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:48 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1509 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:49.438127 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3e8292d7-4c5a-4fa2-ab60-d84140f78e88 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:49.439475 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3e8292d7-4c5a-4fa2-ab60-d84140f78e88 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:49.446446 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3e8292d7-4c5a-4fa2-ab60-d84140f78e88 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:49.447142 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3e8292d7-4c5a-4fa2-ab60-d84140f78e88 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:49.448081 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3e8292d7-4c5a-4fa2-ab60-d84140f78e88 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:49.448697 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3e8292d7-4c5a-4fa2-ab60-d84140f78e88 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:49.482981 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:UPDATE=2,nova_cell1:SELECT=96,nova_cell1:INSERT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:49.621209 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-3e8292d7-4c5a-4fa2-ab60-d84140f78e88 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:49.629997 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3e8292d7-4c5a-4fa2-ab60-d84140f78e88 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1509 microversion: 2.1 time: 0.196207 Jul 05 10:12:49.630587 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 178/387] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:49 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1509 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:50.648672 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a7d176a9-ba20-49b4-b452-43f6c3c72713 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:50.649292 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a7d176a9-ba20-49b4-b452-43f6c3c72713 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:50.655617 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a7d176a9-ba20-49b4-b452-43f6c3c72713 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:50.656062 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a7d176a9-ba20-49b4-b452-43f6c3c72713 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:50.656886 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a7d176a9-ba20-49b4-b452-43f6c3c72713 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:50.657316 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a7d176a9-ba20-49b4-b452-43f6c3c72713 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:50.791551 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a7d176a9-ba20-49b4-b452-43f6c3c72713 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:50.799415 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a7d176a9-ba20-49b4-b452-43f6c3c72713 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1509 microversion: 2.1 time: 0.155057 Jul 05 10:12:50.800152 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 179/388] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:50 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1509 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:51.816360 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6beb7fd5-d92a-48e2-90e3-dfb9f2544a86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:51.817006 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6beb7fd5-d92a-48e2-90e3-dfb9f2544a86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:51.824113 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6beb7fd5-d92a-48e2-90e3-dfb9f2544a86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:51.824598 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6beb7fd5-d92a-48e2-90e3-dfb9f2544a86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:51.825405 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6beb7fd5-d92a-48e2-90e3-dfb9f2544a86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:51.825837 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6beb7fd5-d92a-48e2-90e3-dfb9f2544a86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:51.964673 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6beb7fd5-d92a-48e2-90e3-dfb9f2544a86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:51.972914 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6beb7fd5-d92a-48e2-90e3-dfb9f2544a86 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1509 microversion: 2.1 time: 0.160499 Jul 05 10:12:51.973481 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 180/389] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:51 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1509 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:12:52.398280 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=4 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:52.993370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fc18675b-0372-4035-a6ff-53f0dad9defb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:12:52.994032 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fc18675b-0372-4035-a6ff-53f0dad9defb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:12:53.012232 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fc18675b-0372-4035-a6ff-53f0dad9defb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:53.012717 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fc18675b-0372-4035-a6ff-53f0dad9defb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:53.013506 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fc18675b-0372-4035-a6ff-53f0dad9defb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:12:53.013962 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fc18675b-0372-4035-a6ff-53f0dad9defb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:12:53.221778 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fc18675b-0372-4035-a6ff-53f0dad9defb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:12:53.230310 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fc18675b-0372-4035-a6ff-53f0dad9defb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1515 microversion: 2.1 time: 0.242924 Jul 05 10:12:53.231376 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 181/390] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:12:52 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1515 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:12:54.440719 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=24 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:12:56.498252 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=16,nova_api:INSERT=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:02.045770 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c8219248-c180-4167-b7ad-a24a8c86f32d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Action: 'create', calling method: .version_select at 0x7feb899ab550>, body: {"keypair": {"name": "tempest-keypair-537711298", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHnoZAysCbmg5dSZ8rfU02iy+Gu1VwNUpUBL4RRQv/OwmMEWc0CVVJ9xRMukui3BcEKFlaYqw9kO9L1CNTEScA5QrollD2BZqspH9fjuoONOJ/jC7dNcQjw9p0vFaXoFDw=="}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:13:02.061811 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-c8219248-c180-4167-b7ad-a24a8c86f32d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Getting quotas for project 3f2199fa0bc54901aa24195a250a9e25. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:13:02.067125 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-c8219248-c180-4167-b7ad-a24a8c86f32d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Getting quotas for user 248946c73cb245f9b8e6fe73e46133cf and project 3f2199fa0bc54901aa24195a250a9e25. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:13:02.090503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-c8219248-c180-4167-b7ad-a24a8c86f32d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.048865 Jul 05 10:13:02.091115 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 182/391] 10.222.0.20 () {60 vars in 1158 bytes} [Tue Jul 5 10:13:02 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:13:03.813351 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=20 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:03.838792 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1682722787", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "9dc39dd5-d322-430b-ba08-aae3d3ccc6d0"}], "security_groups": [{"name": "tempest-securitygroup--465625435"}], "key_name": "tempest-keypair-537711298", "user_data": "}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:13:12.073776 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=4 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:13.850282 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=36,nova_api:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:14.161838 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.network.neutron [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] validate_networks() for None {{(pid=117813) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:13:14.420722 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.network.neutron [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] validate_networks() for [('9dc39dd5-d322-430b-ba08-aae3d3ccc6d0', None, None, None, None, None)] {{(pid=117812) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:13:14.748469 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Flavor limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:13:14.748893 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Image limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:13:14.749370 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Flavor pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:13:14.749783 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Image pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:13:14.750196 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:13:14.759597 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:14.760049 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:14.760874 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:14.761310 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:14.778896 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Getting quotas for project 9bf4c2e6ff4347f5b5e46efe5d431dc9. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:13:14.782983 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Getting quotas for user aa8cee3ae343415aae94ce2592a04dd3 and project 9bf4c2e6ff4347f5b5e46efe5d431dc9. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:13:14.794510 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Going to run 1 instances... {{(pid=117813) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:13:14.796918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117813) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:13:14.844257 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-99aaffe7-c22e-434b-9c25-d10c3ad1ccfb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 28.360249 Jul 05 10:13:14.845272 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 210/392] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:12:46 2022] POST /compute/v2.1/servers => generated 380 bytes in 28362 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:13:14.854815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e9445100-2c4b-42e8-aafb-9f91dfe9f293 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:14.855421 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e9445100-2c4b-42e8-aafb-9f91dfe9f293 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:14.928573 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e9445100-2c4b-42e8-aafb-9f91dfe9f293 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:14.937881 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e9445100-2c4b-42e8-aafb-9f91dfe9f293 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1215 microversion: 2.1 time: 0.086672 Jul 05 10:13:14.940502 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 211/393] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:14 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1215 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:15.092812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Flavor limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:13:15.093372 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Image limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:13:15.093724 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Flavor pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:13:15.094145 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Image pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:13:15.094556 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:13:15.105186 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:15.105645 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:15.106447 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:15.106893 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:15.126988 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Getting quotas for project 3f2199fa0bc54901aa24195a250a9e25. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:13:15.131664 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Getting quotas for user 248946c73cb245f9b8e6fe73e46133cf and project 3f2199fa0bc54901aa24195a250a9e25. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:13:15.142714 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Going to run 1 instances... {{(pid=117812) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:13:15.145007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117812) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:13:15.194810 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-801f1184-87aa-442c-85c0-60f8dab9ca2c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 11.382581 Jul 05 10:13:15.195858 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 183/394] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:13:03 2022] POST /compute/v2.1/servers => generated 408 bytes in 11384 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 05 10:13:15.205605 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-64c615f8-2de1-4746-bf33-903698fdd17c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:15.206311 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-64c615f8-2de1-4746-bf33-903698fdd17c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:15.266527 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-64c615f8-2de1-4746-bf33-903698fdd17c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:15.273328 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-64c615f8-2de1-4746-bf33-903698fdd17c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1245 microversion: 2.1 time: 0.071638 Jul 05 10:13:15.273896 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 212/395] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:15 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1245 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:15.955481 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-583f6099-d39a-4356-bb5a-649f038db8cb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:15.956090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-583f6099-d39a-4356-bb5a-649f038db8cb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:15.963664 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-583f6099-d39a-4356-bb5a-649f038db8cb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:15.964114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-583f6099-d39a-4356-bb5a-649f038db8cb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:15.964904 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-583f6099-d39a-4356-bb5a-649f038db8cb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:15.965342 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-583f6099-d39a-4356-bb5a-649f038db8cb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:16.024099 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-583f6099-d39a-4356-bb5a-649f038db8cb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:16.031320 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-583f6099-d39a-4356-bb5a-649f038db8cb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1219 microversion: 2.1 time: 0.080237 Jul 05 10:13:16.031873 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 184/396] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:15 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1219 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:16.290678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-72912d1c-c482-4af7-8a85-a25b7539a3be tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:16.291283 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-72912d1c-c482-4af7-8a85-a25b7539a3be tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:16.298305 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-72912d1c-c482-4af7-8a85-a25b7539a3be tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:16.298782 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-72912d1c-c482-4af7-8a85-a25b7539a3be tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:16.299592 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-72912d1c-c482-4af7-8a85-a25b7539a3be tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:16.300032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-72912d1c-c482-4af7-8a85-a25b7539a3be tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:16.361501 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-72912d1c-c482-4af7-8a85-a25b7539a3be tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:16.368780 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-72912d1c-c482-4af7-8a85-a25b7539a3be tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1249 microversion: 2.1 time: 0.082462 Jul 05 10:13:16.369343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 213/397] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:16 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1249 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:17.047980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-bf3756ee-f3d1-41c1-8235-b5fa219728e7 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:17.048686 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bf3756ee-f3d1-41c1-8235-b5fa219728e7 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:17.055092 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bf3756ee-f3d1-41c1-8235-b5fa219728e7 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:17.055550 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bf3756ee-f3d1-41c1-8235-b5fa219728e7 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:17.056315 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bf3756ee-f3d1-41c1-8235-b5fa219728e7 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:17.056782 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bf3756ee-f3d1-41c1-8235-b5fa219728e7 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:17.119225 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-bf3756ee-f3d1-41c1-8235-b5fa219728e7 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:17.126519 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-bf3756ee-f3d1-41c1-8235-b5fa219728e7 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1211 microversion: 2.1 time: 0.082349 Jul 05 10:13:17.127083 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 185/398] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:17 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1211 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:17.385881 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-42b3d398-d21c-4314-af50-bc246951c5ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:17.386425 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-42b3d398-d21c-4314-af50-bc246951c5ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:17.392362 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-42b3d398-d21c-4314-af50-bc246951c5ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:17.392838 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-42b3d398-d21c-4314-af50-bc246951c5ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:17.393591 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-42b3d398-d21c-4314-af50-bc246951c5ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:17.394026 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-42b3d398-d21c-4314-af50-bc246951c5ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:17.454806 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-42b3d398-d21c-4314-af50-bc246951c5ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:17.463181 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-42b3d398-d21c-4314-af50-bc246951c5ae tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1249 microversion: 2.1 time: 0.081156 Jul 05 10:13:17.463745 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 214/399] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:17 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1249 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:18.153187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-62314870-6ade-4d4d-a02b-f111d3a35732 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:18.153812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-62314870-6ade-4d4d-a02b-f111d3a35732 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:18.159996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-62314870-6ade-4d4d-a02b-f111d3a35732 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:18.164449 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-62314870-6ade-4d4d-a02b-f111d3a35732 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:18.165229 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-62314870-6ade-4d4d-a02b-f111d3a35732 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:18.165680 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-62314870-6ade-4d4d-a02b-f111d3a35732 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:18.229906 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-62314870-6ade-4d4d-a02b-f111d3a35732 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:18.238350 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-62314870-6ade-4d4d-a02b-f111d3a35732 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1211 microversion: 2.1 time: 0.098934 Jul 05 10:13:18.241014 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 186/400] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:18 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1211 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:18.479872 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e19973f4-9d83-4495-9b0c-1de1c36712a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:18.480734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e19973f4-9d83-4495-9b0c-1de1c36712a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:18.486982 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e19973f4-9d83-4495-9b0c-1de1c36712a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:18.487529 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e19973f4-9d83-4495-9b0c-1de1c36712a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:18.488516 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e19973f4-9d83-4495-9b0c-1de1c36712a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:18.488955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e19973f4-9d83-4495-9b0c-1de1c36712a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:18.550243 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e19973f4-9d83-4495-9b0c-1de1c36712a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:18.557600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e19973f4-9d83-4495-9b0c-1de1c36712a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1297 microversion: 2.1 time: 0.082349 Jul 05 10:13:18.558154 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 215/401] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:18 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1297 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:19.264226 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-80b4f09e-4226-43e9-9ef5-7a3bd80f8abd tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:19.264811 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-80b4f09e-4226-43e9-9ef5-7a3bd80f8abd tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:19.271640 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-80b4f09e-4226-43e9-9ef5-7a3bd80f8abd tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:19.272117 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-80b4f09e-4226-43e9-9ef5-7a3bd80f8abd tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:19.273120 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-80b4f09e-4226-43e9-9ef5-7a3bd80f8abd tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:19.273649 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-80b4f09e-4226-43e9-9ef5-7a3bd80f8abd tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:19.359257 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-80b4f09e-4226-43e9-9ef5-7a3bd80f8abd tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:19.384790 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-80b4f09e-4226-43e9-9ef5-7a3bd80f8abd tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1267 microversion: 2.1 time: 0.123909 Jul 05 10:13:19.388469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 187/402] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:19 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1267 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:19.575269 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ab56ff2c-edc9-4d16-8394-b8303245dd33 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:19.576501 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ab56ff2c-edc9-4d16-8394-b8303245dd33 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:19.588102 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ab56ff2c-edc9-4d16-8394-b8303245dd33 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:19.589641 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ab56ff2c-edc9-4d16-8394-b8303245dd33 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:19.596261 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ab56ff2c-edc9-4d16-8394-b8303245dd33 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:19.597437 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ab56ff2c-edc9-4d16-8394-b8303245dd33 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:19.709315 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ab56ff2c-edc9-4d16-8394-b8303245dd33 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:19.721029 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ab56ff2c-edc9-4d16-8394-b8303245dd33 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1303 microversion: 2.1 time: 0.150075 Jul 05 10:13:19.721608 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 216/403] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:19 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1303 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:20.403668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b30b5c41-116f-43b3-a73c-6a85b34a85ef tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:20.404318 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b30b5c41-116f-43b3-a73c-6a85b34a85ef tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:20.411256 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b30b5c41-116f-43b3-a73c-6a85b34a85ef tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:20.411716 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b30b5c41-116f-43b3-a73c-6a85b34a85ef tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:20.413036 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b30b5c41-116f-43b3-a73c-6a85b34a85ef tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:20.413495 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b30b5c41-116f-43b3-a73c-6a85b34a85ef tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:20.521433 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b30b5c41-116f-43b3-a73c-6a85b34a85ef tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:20.529457 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b30b5c41-116f-43b3-a73c-6a85b34a85ef tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1273 microversion: 2.1 time: 0.129658 Jul 05 10:13:20.530027 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 188/404] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:20 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1273 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:20.738719 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f26c1255-2023-4bcd-976f-406cb0f2c877 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:20.739700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f26c1255-2023-4bcd-976f-406cb0f2c877 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:20.748235 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f26c1255-2023-4bcd-976f-406cb0f2c877 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:20.748875 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f26c1255-2023-4bcd-976f-406cb0f2c877 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:20.749821 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f26c1255-2023-4bcd-976f-406cb0f2c877 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:20.750552 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f26c1255-2023-4bcd-976f-406cb0f2c877 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:20.870269 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f26c1255-2023-4bcd-976f-406cb0f2c877 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:20.890499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f26c1255-2023-4bcd-976f-406cb0f2c877 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1303 microversion: 2.1 time: 0.154358 Jul 05 10:13:20.898800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 217/405] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:20 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1303 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:21.547425 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0981f6ec-4ed3-453f-ac97-00b464cd6485 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:21.548707 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0981f6ec-4ed3-453f-ac97-00b464cd6485 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:21.559787 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0981f6ec-4ed3-453f-ac97-00b464cd6485 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:21.560241 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0981f6ec-4ed3-453f-ac97-00b464cd6485 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:21.561051 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0981f6ec-4ed3-453f-ac97-00b464cd6485 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:21.561493 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0981f6ec-4ed3-453f-ac97-00b464cd6485 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:21.914639 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2ff6c24d-a3c9-442e-bb2f-7800fbdabbe5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:21.917067 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2ff6c24d-a3c9-442e-bb2f-7800fbdabbe5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:21.937444 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2ff6c24d-a3c9-442e-bb2f-7800fbdabbe5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:21.937895 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2ff6c24d-a3c9-442e-bb2f-7800fbdabbe5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:21.938669 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2ff6c24d-a3c9-442e-bb2f-7800fbdabbe5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:21.939104 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2ff6c24d-a3c9-442e-bb2f-7800fbdabbe5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:21.946606 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-0981f6ec-4ed3-453f-ac97-00b464cd6485 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:21.954197 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0981f6ec-4ed3-453f-ac97-00b464cd6485 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1315 microversion: 2.1 time: 0.410892 Jul 05 10:13:21.955630 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 189/406] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:21 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1315 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:22.299206 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2ff6c24d-a3c9-442e-bb2f-7800fbdabbe5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:22.313725 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2ff6c24d-a3c9-442e-bb2f-7800fbdabbe5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.405347 Jul 05 10:13:22.314733 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 218/407] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:21 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:22.859842 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-ea487c13-8606-4a76-9075-00a1d4fc9112 req-f2145782-3a94-46d1-b74e-7e1aa89003e5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "12e7fd00-8f3e-4fc4-882b-6ec193de0bfd", "tag": "e27d9ee6-1685-4aec-bb2d-ae3b3206c610"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:13:22.869028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-ea487c13-8606-4a76-9075-00a1d4fc9112 req-f2145782-3a94-46d1-b74e-7e1aa89003e5 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:22.869700 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-ea487c13-8606-4a76-9075-00a1d4fc9112 req-f2145782-3a94-46d1-b74e-7e1aa89003e5 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:22.883304 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-ea487c13-8606-4a76-9075-00a1d4fc9112 req-f2145782-3a94-46d1-b74e-7e1aa89003e5 service nova] Creating event network-changed:e27d9ee6-1685-4aec-bb2d-ae3b3206c610 for instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd on hv-cinder-maste Jul 05 10:13:22.891540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-ea487c13-8606-4a76-9075-00a1d4fc9112 req-f2145782-3a94-46d1-b74e-7e1aa89003e5 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032662 Jul 05 10:13:22.895273 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 190/408] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:13:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:13:22.971290 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f8031409-2883-492e-ab84-acd5a2980acf tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:22.973745 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f8031409-2883-492e-ab84-acd5a2980acf tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:22.986965 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f8031409-2883-492e-ab84-acd5a2980acf tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:22.988360 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f8031409-2883-492e-ab84-acd5a2980acf tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:22.989180 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f8031409-2883-492e-ab84-acd5a2980acf tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:22.993149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f8031409-2883-492e-ab84-acd5a2980acf tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:23.334039 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f6f424f0-b530-4ed3-8baf-174e2cb28132 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:23.335246 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f6f424f0-b530-4ed3-8baf-174e2cb28132 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:23.347847 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f6f424f0-b530-4ed3-8baf-174e2cb28132 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:23.348905 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f6f424f0-b530-4ed3-8baf-174e2cb28132 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:23.353122 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f6f424f0-b530-4ed3-8baf-174e2cb28132 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:23.353885 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f6f424f0-b530-4ed3-8baf-174e2cb28132 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:23.852383 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f8031409-2883-492e-ab84-acd5a2980acf tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:23.868921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f8031409-2883-492e-ab84-acd5a2980acf tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1315 microversion: 2.1 time: 0.901443 Jul 05 10:13:23.877903 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 219/409] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:22 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1315 bytes in 903 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:23.884601 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f6f424f0-b530-4ed3-8baf-174e2cb28132 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:23.892817 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f6f424f0-b530-4ed3-8baf-174e2cb28132 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.564313 Jul 05 10:13:23.895649 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 191/410] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:23 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:24.885640 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5e9e83ec-13b3-4b1c-a546-bef79cfca51f tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:24.900694 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5e9e83ec-13b3-4b1c-a546-bef79cfca51f tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:24.900694 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:24.900694 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5e9e83ec-13b3-4b1c-a546-bef79cfca51f tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:24.900694 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5e9e83ec-13b3-4b1c-a546-bef79cfca51f tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:24.900694 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5e9e83ec-13b3-4b1c-a546-bef79cfca51f tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:24.903622 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5e9e83ec-13b3-4b1c-a546-bef79cfca51f tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:24.933982 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-4afa028e-fa92-4e2e-83a6-ff201fc0e864 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:24.935892 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-4afa028e-fa92-4e2e-83a6-ff201fc0e864 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:24.944510 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4afa028e-fa92-4e2e-83a6-ff201fc0e864 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:24.945210 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4afa028e-fa92-4e2e-83a6-ff201fc0e864 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:24.945988 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4afa028e-fa92-4e2e-83a6-ff201fc0e864 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:24.946420 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4afa028e-fa92-4e2e-83a6-ff201fc0e864 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:25.116922 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:25.138727 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=6 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:25.160778 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5e9e83ec-13b3-4b1c-a546-bef79cfca51f tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:25.171611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5e9e83ec-13b3-4b1c-a546-bef79cfca51f tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1315 microversion: 2.1 time: 0.288537 Jul 05 10:13:25.182064 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 220/411] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:24 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1315 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:25.190639 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-65c9e092-b2d8-4cc0-bf21-9889cbe60d52 req-a1724678-c353-4eae-b1b7-2f2ee3c8efc2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "81676d9e-8ae6-4214-8763-1a143ffac94c", "tag": "c4b5a44e-2a0f-4e9f-bcaa-a8472a69f6a5"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:13:25.203123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-65c9e092-b2d8-4cc0-bf21-9889cbe60d52 req-a1724678-c353-4eae-b1b7-2f2ee3c8efc2 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:25.203816 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-65c9e092-b2d8-4cc0-bf21-9889cbe60d52 req-a1724678-c353-4eae-b1b7-2f2ee3c8efc2 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:25.216116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-4afa028e-fa92-4e2e-83a6-ff201fc0e864 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:25.219222 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-65c9e092-b2d8-4cc0-bf21-9889cbe60d52 req-a1724678-c353-4eae-b1b7-2f2ee3c8efc2 service nova] Creating event network-changed:c4b5a44e-2a0f-4e9f-bcaa-a8472a69f6a5 for instance 81676d9e-8ae6-4214-8763-1a143ffac94c on hv-cinder-maste Jul 05 10:13:25.227900 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-4afa028e-fa92-4e2e-83a6-ff201fc0e864 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.310678 Jul 05 10:13:25.228049 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-65c9e092-b2d8-4cc0-bf21-9889cbe60d52 req-a1724678-c353-4eae-b1b7-2f2ee3c8efc2 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041789 Jul 05 10:13:25.233002 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 221/412] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:13:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:13:25.237309 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 192/413] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:24 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:26.186013 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=10 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:26.190341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-4d24ae6f-c393-47e0-8ed3-3aeb673b1fc6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:26.191040 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-4d24ae6f-c393-47e0-8ed3-3aeb673b1fc6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:26.198054 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4d24ae6f-c393-47e0-8ed3-3aeb673b1fc6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:26.198508 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4d24ae6f-c393-47e0-8ed3-3aeb673b1fc6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:26.199277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4d24ae6f-c393-47e0-8ed3-3aeb673b1fc6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:26.199716 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4d24ae6f-c393-47e0-8ed3-3aeb673b1fc6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:26.246525 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-be72aba6-57e1-40d7-b0d8-3893cd915af1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:26.247133 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-be72aba6-57e1-40d7-b0d8-3893cd915af1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:26.258172 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be72aba6-57e1-40d7-b0d8-3893cd915af1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:26.258632 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be72aba6-57e1-40d7-b0d8-3893cd915af1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:26.259414 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be72aba6-57e1-40d7-b0d8-3893cd915af1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:26.259856 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be72aba6-57e1-40d7-b0d8-3893cd915af1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:26.377410 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4d24ae6f-c393-47e0-8ed3-3aeb673b1fc6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:26.385340 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-4d24ae6f-c393-47e0-8ed3-3aeb673b1fc6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1315 microversion: 2.1 time: 0.200185 Jul 05 10:13:26.394639 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 222/414] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:26 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1315 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:26.432576 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-be72aba6-57e1-40d7-b0d8-3893cd915af1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:26.440622 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-be72aba6-57e1-40d7-b0d8-3893cd915af1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.198469 Jul 05 10:13:26.441180 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 193/415] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:26 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:27.401835 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-fb6e0f4c-a30a-4acb-99f5-9e847311ab6b tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:27.402372 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-fb6e0f4c-a30a-4acb-99f5-9e847311ab6b tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:27.408332 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fb6e0f4c-a30a-4acb-99f5-9e847311ab6b tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:27.408845 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fb6e0f4c-a30a-4acb-99f5-9e847311ab6b tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:27.409606 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fb6e0f4c-a30a-4acb-99f5-9e847311ab6b tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:27.410072 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fb6e0f4c-a30a-4acb-99f5-9e847311ab6b tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:27.457208 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-1cbf3fae-4cae-4e8b-ab09-6d1b45661e93 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:27.457931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-1cbf3fae-4cae-4e8b-ab09-6d1b45661e93 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:27.465218 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1cbf3fae-4cae-4e8b-ab09-6d1b45661e93 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:27.465752 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1cbf3fae-4cae-4e8b-ab09-6d1b45661e93 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:27.466582 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1cbf3fae-4cae-4e8b-ab09-6d1b45661e93 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:27.467083 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1cbf3fae-4cae-4e8b-ab09-6d1b45661e93 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:27.598697 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-fb6e0f4c-a30a-4acb-99f5-9e847311ab6b tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:27.607334 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-fb6e0f4c-a30a-4acb-99f5-9e847311ab6b tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1315 microversion: 2.1 time: 0.208918 Jul 05 10:13:27.611533 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 223/416] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:27 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1315 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:27.630035 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1cbf3fae-4cae-4e8b-ab09-6d1b45661e93 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:27.638071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-1cbf3fae-4cae-4e8b-ab09-6d1b45661e93 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.184716 Jul 05 10:13:27.638815 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 194/417] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:27 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:28.625917 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-010eaba1-87b9-4c08-87c4-36540fec71bb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:28.626521 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-010eaba1-87b9-4c08-87c4-36540fec71bb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:28.633187 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-010eaba1-87b9-4c08-87c4-36540fec71bb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:28.633638 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-010eaba1-87b9-4c08-87c4-36540fec71bb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:28.634402 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-010eaba1-87b9-4c08-87c4-36540fec71bb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:28.634862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-010eaba1-87b9-4c08-87c4-36540fec71bb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:28.655900 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3436ad90-393c-471a-a8bb-01bbd15e0b49 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:28.656684 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3436ad90-393c-471a-a8bb-01bbd15e0b49 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:28.662817 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3436ad90-393c-471a-a8bb-01bbd15e0b49 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:28.663270 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3436ad90-393c-471a-a8bb-01bbd15e0b49 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:28.664037 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3436ad90-393c-471a-a8bb-01bbd15e0b49 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:28.664507 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3436ad90-393c-471a-a8bb-01bbd15e0b49 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:28.842674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-010eaba1-87b9-4c08-87c4-36540fec71bb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:28.846916 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-3436ad90-393c-471a-a8bb-01bbd15e0b49 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:28.849930 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-010eaba1-87b9-4c08-87c4-36540fec71bb tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1315 microversion: 2.1 time: 0.227894 Jul 05 10:13:28.852545 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 224/418] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:28 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1315 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:28.854337 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3436ad90-393c-471a-a8bb-01bbd15e0b49 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.203110 Jul 05 10:13:28.854977 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 195/419] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:28 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:29.878697 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-081c1b3d-414a-4bfc-b00e-138058ae675e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:29.881064 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-081c1b3d-414a-4bfc-b00e-138058ae675e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:29.882528 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1e057b30-1e7b-4f0e-92a4-aab098c40ab4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:29.884129 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1e057b30-1e7b-4f0e-92a4-aab098c40ab4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:29.902481 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-081c1b3d-414a-4bfc-b00e-138058ae675e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:29.903813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-081c1b3d-414a-4bfc-b00e-138058ae675e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:29.904523 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1e057b30-1e7b-4f0e-92a4-aab098c40ab4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:29.906875 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-081c1b3d-414a-4bfc-b00e-138058ae675e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:29.907333 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1e057b30-1e7b-4f0e-92a4-aab098c40ab4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:29.908363 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1e057b30-1e7b-4f0e-92a4-aab098c40ab4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:29.909039 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-081c1b3d-414a-4bfc-b00e-138058ae675e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:29.913372 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1e057b30-1e7b-4f0e-92a4-aab098c40ab4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:30.162354 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1e057b30-1e7b-4f0e-92a4-aab098c40ab4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:30.176563 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1e057b30-1e7b-4f0e-92a4-aab098c40ab4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1315 microversion: 2.1 time: 0.313372 Jul 05 10:13:30.181600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 225/420] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:29 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1315 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:30.201455 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-081c1b3d-414a-4bfc-b00e-138058ae675e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:30.209562 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-081c1b3d-414a-4bfc-b00e-138058ae675e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.340863 Jul 05 10:13:30.210975 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 196/421] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:29 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:31.197085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c844337a-7e8b-4945-b83d-0363d22cb047 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:31.197700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c844337a-7e8b-4945-b83d-0363d22cb047 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:31.203996 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c844337a-7e8b-4945-b83d-0363d22cb047 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:31.204712 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c844337a-7e8b-4945-b83d-0363d22cb047 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:31.205472 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c844337a-7e8b-4945-b83d-0363d22cb047 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:31.205904 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c844337a-7e8b-4945-b83d-0363d22cb047 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:31.225941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ceb508b3-9af9-4735-9ea0-09ca0dbcf5fb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:31.226689 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ceb508b3-9af9-4735-9ea0-09ca0dbcf5fb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:31.232908 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ceb508b3-9af9-4735-9ea0-09ca0dbcf5fb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:31.233470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ceb508b3-9af9-4735-9ea0-09ca0dbcf5fb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:31.234462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ceb508b3-9af9-4735-9ea0-09ca0dbcf5fb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:31.234989 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ceb508b3-9af9-4735-9ea0-09ca0dbcf5fb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:31.451039 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c844337a-7e8b-4945-b83d-0363d22cb047 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:31.460902 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c844337a-7e8b-4945-b83d-0363d22cb047 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1315 microversion: 2.1 time: 0.268058 Jul 05 10:13:31.462415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 226/422] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:31 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1315 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:31.474835 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ceb508b3-9af9-4735-9ea0-09ca0dbcf5fb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:31.485905 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ceb508b3-9af9-4735-9ea0-09ca0dbcf5fb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.263661 Jul 05 10:13:31.496474 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 197/423] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:31 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:32.478156 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-aa9185d3-bd7d-43fc-bc5f-8975ff6d30d6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:32.478708 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-aa9185d3-bd7d-43fc-bc5f-8975ff6d30d6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:32.485104 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-aa9185d3-bd7d-43fc-bc5f-8975ff6d30d6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:32.485727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-aa9185d3-bd7d-43fc-bc5f-8975ff6d30d6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:32.486494 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-aa9185d3-bd7d-43fc-bc5f-8975ff6d30d6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:32.486932 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-aa9185d3-bd7d-43fc-bc5f-8975ff6d30d6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:32.503022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-27abc798-8364-490f-b9bd-76201ff64607 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:32.503624 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-27abc798-8364-490f-b9bd-76201ff64607 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:32.512261 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-27abc798-8364-490f-b9bd-76201ff64607 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:32.512737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-27abc798-8364-490f-b9bd-76201ff64607 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:32.513512 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-27abc798-8364-490f-b9bd-76201ff64607 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:32.513947 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-27abc798-8364-490f-b9bd-76201ff64607 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:32.679914 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-aa9185d3-bd7d-43fc-bc5f-8975ff6d30d6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:32.690371 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-aa9185d3-bd7d-43fc-bc5f-8975ff6d30d6 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1315 microversion: 2.1 time: 0.216428 Jul 05 10:13:32.691355 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 227/424] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:32 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1315 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:32.706995 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-27abc798-8364-490f-b9bd-76201ff64607 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:32.715080 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-27abc798-8364-490f-b9bd-76201ff64607 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.215814 Jul 05 10:13:32.715644 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 198/425] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:32 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:33.242257 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-6a1f9e02-505b-4c42-9222-198dbe51e310 req-3a5bedf5-f0c1-4a28-a941-528c18b754c7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "81676d9e-8ae6-4214-8763-1a143ffac94c", "name": "network-vif-plugged", "status": "completed", "tag": "c4b5a44e-2a0f-4e9f-bcaa-a8472a69f6a5"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:13:33.250389 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-6a1f9e02-505b-4c42-9222-198dbe51e310 req-3a5bedf5-f0c1-4a28-a941-528c18b754c7 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:33.250833 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-6a1f9e02-505b-4c42-9222-198dbe51e310 req-3a5bedf5-f0c1-4a28-a941-528c18b754c7 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:33.264251 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-6a1f9e02-505b-4c42-9222-198dbe51e310 req-3a5bedf5-f0c1-4a28-a941-528c18b754c7 service nova] Creating event network-vif-plugged:c4b5a44e-2a0f-4e9f-bcaa-a8472a69f6a5 for instance 81676d9e-8ae6-4214-8763-1a143ffac94c on hv-cinder-maste Jul 05 10:13:33.269238 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-6a1f9e02-505b-4c42-9222-198dbe51e310 req-3a5bedf5-f0c1-4a28-a941-528c18b754c7 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031269 Jul 05 10:13:33.269838 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 228/426] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:13:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:13:33.707214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-687aa421-63aa-47f7-8621-2614279c0b0c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:33.707833 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-687aa421-63aa-47f7-8621-2614279c0b0c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:33.714656 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-687aa421-63aa-47f7-8621-2614279c0b0c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:33.715106 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-687aa421-63aa-47f7-8621-2614279c0b0c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:33.715874 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-687aa421-63aa-47f7-8621-2614279c0b0c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:33.716313 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-687aa421-63aa-47f7-8621-2614279c0b0c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:33.731523 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-da04368c-5ab1-490f-afbe-cb0d860005d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:33.732212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-da04368c-5ab1-490f-afbe-cb0d860005d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:33.738315 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-da04368c-5ab1-490f-afbe-cb0d860005d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:33.738759 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-da04368c-5ab1-490f-afbe-cb0d860005d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:33.739734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-da04368c-5ab1-490f-afbe-cb0d860005d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:33.742975 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-da04368c-5ab1-490f-afbe-cb0d860005d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:33.906385 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-687aa421-63aa-47f7-8621-2614279c0b0c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:33.915090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-687aa421-63aa-47f7-8621-2614279c0b0c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1315 microversion: 2.1 time: 0.211707 Jul 05 10:13:33.915732 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 199/427] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:33 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1315 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:33.933747 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-da04368c-5ab1-490f-afbe-cb0d860005d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:33.941603 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-da04368c-5ab1-490f-afbe-cb0d860005d9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.214201 Jul 05 10:13:33.942089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 229/428] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:33 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:34.686133 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1696130622", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "networks": [{"uuid": "11e0e390-3962-4c1a-9a29-d936175172f2"}]}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:13:34.739767 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.network.neutron [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] validate_networks() for [('11e0e390-3962-4c1a-9a29-d936175172f2', None, None, None, None, None)] {{(pid=117812) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:13:34.940708 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b3b239f7-f170-4629-b559-ca4bcc65d98c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:34.941320 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b3b239f7-f170-4629-b559-ca4bcc65d98c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:34.948220 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b3b239f7-f170-4629-b559-ca4bcc65d98c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:34.948720 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b3b239f7-f170-4629-b559-ca4bcc65d98c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:34.949566 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b3b239f7-f170-4629-b559-ca4bcc65d98c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:34.950008 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b3b239f7-f170-4629-b559-ca4bcc65d98c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:34.969485 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=100 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:35.380192 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b3b239f7-f170-4629-b559-ca4bcc65d98c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:35.388538 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b3b239f7-f170-4629-b559-ca4bcc65d98c tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1315 microversion: 2.1 time: 0.453731 Jul 05 10:13:35.389116 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 230/429] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:34 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1315 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:35.394216 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e8198ee6-849b-4462-bbf3-013abfa8b624 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:35.394821 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e8198ee6-849b-4462-bbf3-013abfa8b624 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:35.401404 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e8198ee6-849b-4462-bbf3-013abfa8b624 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:35.402042 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e8198ee6-849b-4462-bbf3-013abfa8b624 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:35.403597 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e8198ee6-849b-4462-bbf3-013abfa8b624 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:35.404037 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e8198ee6-849b-4462-bbf3-013abfa8b624 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:35.634408 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e8198ee6-849b-4462-bbf3-013abfa8b624 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:35.647652 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e8198ee6-849b-4462-bbf3-013abfa8b624 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.256847 Jul 05 10:13:35.656962 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 231/430] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:35 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:35.674148 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Flavor limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:13:35.674719 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Image limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:13:35.675244 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Flavor pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:13:35.675657 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Image pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:13:35.676068 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:13:35.693068 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:35.693523 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:35.694321 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:35.697732 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:35.716641 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Getting quotas for project accabbba722345d190b52446d1e7c2b7. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:13:35.720655 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Getting quotas for user df495d1fbab24736a2ba12e8a0e7d803 and project accabbba722345d190b52446d1e7c2b7. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:13:35.732583 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Going to run 1 instances... {{(pid=117812) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:13:35.734854 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117812) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:13:35.780904 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2bb490d5-e0d1-4066-a27a-c29218a01a7a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.098488 Jul 05 10:13:35.781545 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 200/431] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:13:34 2022] POST /compute/v2.1/servers => generated 380 bytes in 1100 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 05 10:13:35.791631 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-55929120-413c-4716-9549-e1d72fd089a9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:35.793222 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-55929120-413c-4716-9549-e1d72fd089a9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:35.869689 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-55929120-413c-4716-9549-e1d72fd089a9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:35.890158 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-55929120-413c-4716-9549-e1d72fd089a9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1219 microversion: 2.1 time: 0.102422 Jul 05 10:13:35.892521 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 232/432] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:35 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1219 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:35.982577 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1126831344", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-309513784", "security_groups": [{"name": "tempest-secgroup-smoke-142710623"}], "networks": [{"uuid": "f5e3c29d-a64e-4a6c-9aa8-92b19a878a3c"}], "block_device_mapping_v2": [{"uuid": "d9da3ab3-8e88-450a-a0ee-e741a107b586", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:13:36.121936 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.network.neutron [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] validate_networks() for [('f5e3c29d-a64e-4a6c-9aa8-92b19a878a3c', None, None, None, None, None)] {{(pid=117812) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:13:36.404951 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0817e388-7110-4bb9-96ff-bd69e96afe99 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:36.406166 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0817e388-7110-4bb9-96ff-bd69e96afe99 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:36.412680 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0817e388-7110-4bb9-96ff-bd69e96afe99 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:36.414077 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0817e388-7110-4bb9-96ff-bd69e96afe99 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:36.415348 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0817e388-7110-4bb9-96ff-bd69e96afe99 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:36.415999 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0817e388-7110-4bb9-96ff-bd69e96afe99 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:36.658618 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Flavor limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:13:36.659158 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Image limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:13:36.659774 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Flavor pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:13:36.660078 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Image pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:13:36.660517 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:13:36.663110 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0817e388-7110-4bb9-96ff-bd69e96afe99 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:36.669810 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:36.670259 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:36.671048 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:36.671484 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:36.677867 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0817e388-7110-4bb9-96ff-bd69e96afe99 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1315 microversion: 2.1 time: 0.276963 Jul 05 10:13:36.688063 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 233/433] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:36 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1315 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:36.690957 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Getting quotas for project 9edab973767545988e178ddff76557ac. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:13:36.696793 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-688f1860-8ae5-4df3-bdd3-e34595da5577 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:36.697591 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-688f1860-8ae5-4df3-bdd3-e34595da5577 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:36.699100 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Getting quotas for user 7e322227b42b4aaa99674afd906a706a and project 9edab973767545988e178ddff76557ac. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:13:36.700147 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=64,nova_api:INSERT=3 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:36.709467 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-688f1860-8ae5-4df3-bdd3-e34595da5577 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:36.709914 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-688f1860-8ae5-4df3-bdd3-e34595da5577 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:36.711568 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-688f1860-8ae5-4df3-bdd3-e34595da5577 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:36.712174 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-688f1860-8ae5-4df3-bdd3-e34595da5577 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:36.804664 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Going to run 1 instances... {{(pid=117812) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:13:36.807206 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='d9da3ab3-8e88-450a-a0ee-e741a107b586',volume_size=None,volume_type=None)] {{(pid=117812) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:13:36.905336 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-688f1860-8ae5-4df3-bdd3-e34595da5577 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:36.915157 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-688f1860-8ae5-4df3-bdd3-e34595da5577 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.224398 Jul 05 10:13:36.932385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 234/434] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:36 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:36.934798 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-77f5ac87-fa50-456c-9528-00fd52cc9683 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:36.935420 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-77f5ac87-fa50-456c-9528-00fd52cc9683 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:36.947597 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-77f5ac87-fa50-456c-9528-00fd52cc9683 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:36.948153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-77f5ac87-fa50-456c-9528-00fd52cc9683 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:36.948913 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-77f5ac87-fa50-456c-9528-00fd52cc9683 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:36.949352 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-77f5ac87-fa50-456c-9528-00fd52cc9683 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:37.032769 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a166fde2-11ce-4a84-9f90-9b88f46a4337 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.054808 Jul 05 10:13:37.034481 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 201/435] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:13:35 2022] POST /compute/v2.1/servers => generated 405 bytes in 1057 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:13:37.050605 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fd6a61af-2177-4343-8ba6-a80dcaf3c8ce tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:37.051536 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-77f5ac87-fa50-456c-9528-00fd52cc9683 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:37.052518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fd6a61af-2177-4343-8ba6-a80dcaf3c8ce tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:37.059856 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-77f5ac87-fa50-456c-9528-00fd52cc9683 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1271 microversion: 2.1 time: 0.134596 Jul 05 10:13:37.070967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 235/436] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:36 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1271 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:37.126697 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fd6a61af-2177-4343-8ba6-a80dcaf3c8ce tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:37.135015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fd6a61af-2177-4343-8ba6-a80dcaf3c8ce tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1088 microversion: 2.1 time: 0.094646 Jul 05 10:13:37.135924 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 202/437] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:37 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1088 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:37.448656 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-d63655c2-9c3c-403a-8593-46a93d5d08d6 req-eee2b78c-2cdf-4b89-a499-aaae15280ad9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "12e7fd00-8f3e-4fc4-882b-6ec193de0bfd", "name": "network-vif-plugged", "status": "completed", "tag": "e27d9ee6-1685-4aec-bb2d-ae3b3206c610"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:13:37.459354 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-d63655c2-9c3c-403a-8593-46a93d5d08d6 req-eee2b78c-2cdf-4b89-a499-aaae15280ad9 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:37.460921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-d63655c2-9c3c-403a-8593-46a93d5d08d6 req-eee2b78c-2cdf-4b89-a499-aaae15280ad9 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:37.479120 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-d63655c2-9c3c-403a-8593-46a93d5d08d6 req-eee2b78c-2cdf-4b89-a499-aaae15280ad9 service nova] Creating event network-vif-plugged:e27d9ee6-1685-4aec-bb2d-ae3b3206c610 for instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd on hv-cinder-maste Jul 05 10:13:37.489069 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-d63655c2-9c3c-403a-8593-46a93d5d08d6 req-eee2b78c-2cdf-4b89-a499-aaae15280ad9 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046346 Jul 05 10:13:37.490004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 236/438] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:13:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:13:37.694174 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a4558ba0-ad8e-4f02-8a67-862a9593ff19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:37.694829 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a4558ba0-ad8e-4f02-8a67-862a9593ff19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:37.700876 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a4558ba0-ad8e-4f02-8a67-862a9593ff19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:37.701355 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a4558ba0-ad8e-4f02-8a67-862a9593ff19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:37.702124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a4558ba0-ad8e-4f02-8a67-862a9593ff19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:37.702576 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a4558ba0-ad8e-4f02-8a67-862a9593ff19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:37.706809 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=100 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:37.840067 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a4558ba0-ad8e-4f02-8a67-862a9593ff19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:37.847604 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a4558ba0-ad8e-4f02-8a67-862a9593ff19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1465 microversion: 2.1 time: 0.157301 Jul 05 10:13:37.848162 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 203/439] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:37 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1465 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:37.888750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d2ed8b69-84c3-421a-bd8d-3f47792424b2 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:37.889332 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d2ed8b69-84c3-421a-bd8d-3f47792424b2 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:37.895461 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d2ed8b69-84c3-421a-bd8d-3f47792424b2 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:37.895907 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d2ed8b69-84c3-421a-bd8d-3f47792424b2 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:37.896897 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d2ed8b69-84c3-421a-bd8d-3f47792424b2 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:37.897329 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d2ed8b69-84c3-421a-bd8d-3f47792424b2 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:37.924760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d2ed8b69-84c3-421a-bd8d-3f47792424b2 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Going to try to terminate instance {{(pid=117813) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:13:37.940441 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-34070174-b90a-480a-ab09-028d58663f7e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:37.941054 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-34070174-b90a-480a-ab09-028d58663f7e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:37.946943 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-34070174-b90a-480a-ab09-028d58663f7e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:37.947596 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-34070174-b90a-480a-ab09-028d58663f7e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:37.948361 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-34070174-b90a-480a-ab09-028d58663f7e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:37.948818 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-34070174-b90a-480a-ab09-028d58663f7e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:38.018885 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d2ed8b69-84c3-421a-bd8d-3f47792424b2 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "DELETE /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 204 len: 0 microversion: 2.1 time: 0.133972 Jul 05 10:13:38.019985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 237/440] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:13:37 2022] DELETE /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:13:38.029136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-82f610ff-7715-4f2e-a389-0ada2a41a11e tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:38.030008 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-82f610ff-7715-4f2e-a389-0ada2a41a11e tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:38.036879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-82f610ff-7715-4f2e-a389-0ada2a41a11e tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:38.037881 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-82f610ff-7715-4f2e-a389-0ada2a41a11e tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:38.039414 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-82f610ff-7715-4f2e-a389-0ada2a41a11e tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:38.040152 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-82f610ff-7715-4f2e-a389-0ada2a41a11e tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:38.176987 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-34070174-b90a-480a-ab09-028d58663f7e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:38.196265 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-34070174-b90a-480a-ab09-028d58663f7e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.259512 Jul 05 10:13:38.206839 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 204/441] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:37 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:38.214891 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8cd6edeb-f609-4de8-b814-10b9527ff568 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:38.215747 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-8cd6edeb-f609-4de8-b814-10b9527ff568 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:38.223123 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8cd6edeb-f609-4de8-b814-10b9527ff568 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:38.223861 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8cd6edeb-f609-4de8-b814-10b9527ff568 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:38.224896 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8cd6edeb-f609-4de8-b814-10b9527ff568 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:38.225407 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8cd6edeb-f609-4de8-b814-10b9527ff568 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:38.244519 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-82f610ff-7715-4f2e-a389-0ada2a41a11e tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:38.253916 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-82f610ff-7715-4f2e-a389-0ada2a41a11e tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1471 microversion: 2.1 time: 0.228642 Jul 05 10:13:38.264093 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 238/442] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:38 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1471 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:38.269129 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-18e89d03-78fd-454d-9ea8-f32225808035 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:38.269855 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-18e89d03-78fd-454d-9ea8-f32225808035 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:38.276493 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18e89d03-78fd-454d-9ea8-f32225808035 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:38.276974 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18e89d03-78fd-454d-9ea8-f32225808035 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:38.277738 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18e89d03-78fd-454d-9ea8-f32225808035 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:38.278430 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18e89d03-78fd-454d-9ea8-f32225808035 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:38.321445 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8cd6edeb-f609-4de8-b814-10b9527ff568 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:38.328869 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-8cd6edeb-f609-4de8-b814-10b9527ff568 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1289 microversion: 2.1 time: 0.118814 Jul 05 10:13:38.329544 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 205/443] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:38 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1289 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:38.354770 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-18e89d03-78fd-454d-9ea8-f32225808035 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:38.365447 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-18e89d03-78fd-454d-9ea8-f32225808035 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1138 microversion: 2.1 time: 0.100008 Jul 05 10:13:38.372382 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 239/444] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:38 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1138 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:39.213883 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-bbacef06-b22f-41eb-a71c-e0af5a8a03b2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:39.214491 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bbacef06-b22f-41eb-a71c-e0af5a8a03b2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:39.220976 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bbacef06-b22f-41eb-a71c-e0af5a8a03b2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:39.221425 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bbacef06-b22f-41eb-a71c-e0af5a8a03b2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:39.222185 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bbacef06-b22f-41eb-a71c-e0af5a8a03b2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:39.222619 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bbacef06-b22f-41eb-a71c-e0af5a8a03b2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:39.271096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a3d738d4-cd64-4b4b-858e-1ecda528c293 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:39.271869 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a3d738d4-cd64-4b4b-858e-1ecda528c293 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:39.278029 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a3d738d4-cd64-4b4b-858e-1ecda528c293 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:39.278480 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a3d738d4-cd64-4b4b-858e-1ecda528c293 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:39.279241 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a3d738d4-cd64-4b4b-858e-1ecda528c293 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:39.279678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a3d738d4-cd64-4b4b-858e-1ecda528c293 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:39.389555 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-bbacef06-b22f-41eb-a71c-e0af5a8a03b2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:39.397050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-bbacef06-b22f-41eb-a71c-e0af5a8a03b2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.187335 Jul 05 10:13:39.397622 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 206/445] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:39 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:39.401962 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-682eb3eb-1e39-4631-89e3-88803e5aa6b9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:39.402564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-682eb3eb-1e39-4631-89e3-88803e5aa6b9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:39.408553 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-682eb3eb-1e39-4631-89e3-88803e5aa6b9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:39.409001 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-682eb3eb-1e39-4631-89e3-88803e5aa6b9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:39.409764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-682eb3eb-1e39-4631-89e3-88803e5aa6b9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:39.410203 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-682eb3eb-1e39-4631-89e3-88803e5aa6b9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:39.436383 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a3d738d4-cd64-4b4b-858e-1ecda528c293 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:39.444015 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a3d738d4-cd64-4b4b-858e-1ecda528c293 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1471 microversion: 2.1 time: 0.176744 Jul 05 10:13:39.453237 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 240/446] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:39 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1471 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:39.453237 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1a4f262a-6abf-4764-ab40-8b2bf5198817 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:39.453237 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1a4f262a-6abf-4764-ab40-8b2bf5198817 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:39.455426 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1a4f262a-6abf-4764-ab40-8b2bf5198817 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:39.455875 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1a4f262a-6abf-4764-ab40-8b2bf5198817 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:39.456664 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1a4f262a-6abf-4764-ab40-8b2bf5198817 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:39.457103 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1a4f262a-6abf-4764-ab40-8b2bf5198817 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:39.471665 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-682eb3eb-1e39-4631-89e3-88803e5aa6b9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:39.479998 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-682eb3eb-1e39-4631-89e3-88803e5aa6b9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1289 microversion: 2.1 time: 0.081442 Jul 05 10:13:39.480595 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 207/447] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:39 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1289 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:39.517717 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1a4f262a-6abf-4764-ab40-8b2bf5198817 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:39.525546 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1a4f262a-6abf-4764-ab40-8b2bf5198817 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1138 microversion: 2.1 time: 0.079972 Jul 05 10:13:39.526108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 241/448] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:39 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1138 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:40.413792 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8fbff3c5-8adc-406d-9898-588608c4bee6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:40.414413 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-8fbff3c5-8adc-406d-9898-588608c4bee6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:40.420829 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8fbff3c5-8adc-406d-9898-588608c4bee6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:40.421274 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8fbff3c5-8adc-406d-9898-588608c4bee6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:40.422031 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8fbff3c5-8adc-406d-9898-588608c4bee6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:40.422467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8fbff3c5-8adc-406d-9898-588608c4bee6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:40.461008 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ff2124de-1eaf-420a-b446-1a867600dd19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:40.461740 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ff2124de-1eaf-420a-b446-1a867600dd19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:40.467703 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ff2124de-1eaf-420a-b446-1a867600dd19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:40.468151 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ff2124de-1eaf-420a-b446-1a867600dd19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:40.468961 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ff2124de-1eaf-420a-b446-1a867600dd19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:40.469393 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ff2124de-1eaf-420a-b446-1a867600dd19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:40.607169 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8fbff3c5-8adc-406d-9898-588608c4bee6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:40.614538 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-8fbff3c5-8adc-406d-9898-588608c4bee6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.204525 Jul 05 10:13:40.615111 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 208/449] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:40 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:40.620078 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-40b24e8c-3426-4f0c-8e8c-9f39489b0778 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:40.620721 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-40b24e8c-3426-4f0c-8e8c-9f39489b0778 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:40.626571 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:40.628517 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-40b24e8c-3426-4f0c-8e8c-9f39489b0778 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:40.629236 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-40b24e8c-3426-4f0c-8e8c-9f39489b0778 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:40.630058 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-40b24e8c-3426-4f0c-8e8c-9f39489b0778 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:40.630495 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-40b24e8c-3426-4f0c-8e8c-9f39489b0778 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:40.637449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ff2124de-1eaf-420a-b446-1a867600dd19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:40.645754 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ff2124de-1eaf-420a-b446-1a867600dd19 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1471 microversion: 2.1 time: 0.188791 Jul 05 10:13:40.646404 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 242/450] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:40 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1471 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:40.651104 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-581201dd-9ed5-444c-ba10-c51ed91d1ec4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:40.651797 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-581201dd-9ed5-444c-ba10-c51ed91d1ec4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:40.657662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-581201dd-9ed5-444c-ba10-c51ed91d1ec4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:40.658186 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-581201dd-9ed5-444c-ba10-c51ed91d1ec4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:40.659006 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-581201dd-9ed5-444c-ba10-c51ed91d1ec4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:40.659508 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-581201dd-9ed5-444c-ba10-c51ed91d1ec4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:40.700164 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-40b24e8c-3426-4f0c-8e8c-9f39489b0778 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:40.707605 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-40b24e8c-3426-4f0c-8e8c-9f39489b0778 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1289 microversion: 2.1 time: 0.091525 Jul 05 10:13:40.708181 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 209/451] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:40 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1289 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:40.726691 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-581201dd-9ed5-444c-ba10-c51ed91d1ec4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:40.734239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-581201dd-9ed5-444c-ba10-c51ed91d1ec4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1130 microversion: 2.1 time: 0.086731 Jul 05 10:13:40.734790 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 243/452] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:40 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1130 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:41.633980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-253a81dd-4df5-4ea6-b8f0-9d25401b188c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:41.634552 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-253a81dd-4df5-4ea6-b8f0-9d25401b188c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:41.641024 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-253a81dd-4df5-4ea6-b8f0-9d25401b188c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:41.641475 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-253a81dd-4df5-4ea6-b8f0-9d25401b188c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:41.642257 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-253a81dd-4df5-4ea6-b8f0-9d25401b188c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:41.642705 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-253a81dd-4df5-4ea6-b8f0-9d25401b188c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:41.661661 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0cf6b5a7-a568-45a6-82ab-a41e7ca090ea tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:41.662594 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0cf6b5a7-a568-45a6-82ab-a41e7ca090ea tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:41.669068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0cf6b5a7-a568-45a6-82ab-a41e7ca090ea tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:41.669701 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0cf6b5a7-a568-45a6-82ab-a41e7ca090ea tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:41.670649 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0cf6b5a7-a568-45a6-82ab-a41e7ca090ea tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:41.671189 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0cf6b5a7-a568-45a6-82ab-a41e7ca090ea tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:41.841517 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-253a81dd-4df5-4ea6-b8f0-9d25401b188c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:41.846665 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0cf6b5a7-a568-45a6-82ab-a41e7ca090ea tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:41.849038 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-253a81dd-4df5-4ea6-b8f0-9d25401b188c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.219533 Jul 05 10:13:41.849611 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 210/453] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:41 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:41.854235 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0cf6b5a7-a568-45a6-82ab-a41e7ca090ea tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1471 microversion: 2.1 time: 0.196493 Jul 05 10:13:41.858897 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 244/454] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:41 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1471 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:41.859051 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a5c1416f-3dcf-4953-acc0-3da8ed5975bf tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:41.859051 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a5c1416f-3dcf-4953-acc0-3da8ed5975bf tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:41.862357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5c2815ef-fe75-43b9-b1cc-b6bdc45fcbc1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:41.862916 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5c1416f-3dcf-4953-acc0-3da8ed5975bf tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:41.863040 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5c2815ef-fe75-43b9-b1cc-b6bdc45fcbc1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:41.863366 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5c1416f-3dcf-4953-acc0-3da8ed5975bf tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:41.864135 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5c1416f-3dcf-4953-acc0-3da8ed5975bf tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:41.864604 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5c1416f-3dcf-4953-acc0-3da8ed5975bf tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:41.869366 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5c2815ef-fe75-43b9-b1cc-b6bdc45fcbc1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:41.869808 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5c2815ef-fe75-43b9-b1cc-b6bdc45fcbc1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:41.870560 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5c2815ef-fe75-43b9-b1cc-b6bdc45fcbc1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:41.870986 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5c2815ef-fe75-43b9-b1cc-b6bdc45fcbc1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:41.952275 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a5c1416f-3dcf-4953-acc0-3da8ed5975bf tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:41.953900 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5c2815ef-fe75-43b9-b1cc-b6bdc45fcbc1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:41.960105 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a5c1416f-3dcf-4953-acc0-3da8ed5975bf tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1289 microversion: 2.1 time: 0.108050 Jul 05 10:13:41.960991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 211/455] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:41 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1289 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:41.962265 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5c2815ef-fe75-43b9-b1cc-b6bdc45fcbc1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1130 microversion: 2.1 time: 0.106145 Jul 05 10:13:41.962938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 245/456] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:41 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1130 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:42.867355 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0bd96c05-4422-4e74-9e98-e70b071d8834 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:42.867964 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0bd96c05-4422-4e74-9e98-e70b071d8834 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:42.870954 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d6dfc36b-793f-4600-b22d-8a5195f9ea70 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:42.871568 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d6dfc36b-793f-4600-b22d-8a5195f9ea70 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:42.877102 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0bd96c05-4422-4e74-9e98-e70b071d8834 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:42.877240 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0bd96c05-4422-4e74-9e98-e70b071d8834 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:42.877987 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0bd96c05-4422-4e74-9e98-e70b071d8834 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:42.878506 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0bd96c05-4422-4e74-9e98-e70b071d8834 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:42.891387 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d6dfc36b-793f-4600-b22d-8a5195f9ea70 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:42.893778 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d6dfc36b-793f-4600-b22d-8a5195f9ea70 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:42.895947 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d6dfc36b-793f-4600-b22d-8a5195f9ea70 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:42.896421 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d6dfc36b-793f-4600-b22d-8a5195f9ea70 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:43.147492 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-0bd96c05-4422-4e74-9e98-e70b071d8834 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:43.155748 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0bd96c05-4422-4e74-9e98-e70b071d8834 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.293908 Jul 05 10:13:43.160923 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 212/457] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:42 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:43.161281 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8feef3c7-339b-462a-8ff6-afed49c875eb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:43.162076 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-8feef3c7-339b-462a-8ff6-afed49c875eb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:43.163010 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d6dfc36b-793f-4600-b22d-8a5195f9ea70 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:43.168278 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8feef3c7-339b-462a-8ff6-afed49c875eb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:43.168773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8feef3c7-339b-462a-8ff6-afed49c875eb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:43.169533 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8feef3c7-339b-462a-8ff6-afed49c875eb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:43.169976 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8feef3c7-339b-462a-8ff6-afed49c875eb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:43.172455 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d6dfc36b-793f-4600-b22d-8a5195f9ea70 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1471 microversion: 2.1 time: 0.304982 Jul 05 10:13:43.173136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 246/458] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:42 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1471 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:43.178886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8ea3d4ed-9546-4339-b6fc-59d85ba8f985 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:43.180141 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8ea3d4ed-9546-4339-b6fc-59d85ba8f985 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:43.186373 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ea3d4ed-9546-4339-b6fc-59d85ba8f985 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:43.186873 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ea3d4ed-9546-4339-b6fc-59d85ba8f985 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:43.187768 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ea3d4ed-9546-4339-b6fc-59d85ba8f985 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:43.188299 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ea3d4ed-9546-4339-b6fc-59d85ba8f985 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:43.192613 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:43.271625 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8feef3c7-339b-462a-8ff6-afed49c875eb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:43.279019 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-8feef3c7-339b-462a-8ff6-afed49c875eb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1277 microversion: 2.1 time: 0.121680 Jul 05 10:13:43.279794 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 213/459] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:43 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1277 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:43.290608 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8ea3d4ed-9546-4339-b6fc-59d85ba8f985 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:43.298420 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8ea3d4ed-9546-4339-b6fc-59d85ba8f985 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1186 microversion: 2.1 time: 0.123342 Jul 05 10:13:43.308104 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 247/460] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:43 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1186 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:44.172062 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ca567584-3b39-4e38-bdb8-a7a9f644fbd2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:44.172699 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ca567584-3b39-4e38-bdb8-a7a9f644fbd2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:44.179315 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ca567584-3b39-4e38-bdb8-a7a9f644fbd2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:44.179769 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ca567584-3b39-4e38-bdb8-a7a9f644fbd2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:44.180572 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ca567584-3b39-4e38-bdb8-a7a9f644fbd2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:44.181015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ca567584-3b39-4e38-bdb8-a7a9f644fbd2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:44.190342 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-cc9cb51d-55f4-432c-b8f1-104a3357b7c3 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:44.190944 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-cc9cb51d-55f4-432c-b8f1-104a3357b7c3 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:44.196825 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cc9cb51d-55f4-432c-b8f1-104a3357b7c3 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:44.197264 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cc9cb51d-55f4-432c-b8f1-104a3357b7c3 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:44.198021 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cc9cb51d-55f4-432c-b8f1-104a3357b7c3 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:44.198447 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cc9cb51d-55f4-432c-b8f1-104a3357b7c3 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:44.381300 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ca567584-3b39-4e38-bdb8-a7a9f644fbd2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:44.384421 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-cc9cb51d-55f4-432c-b8f1-104a3357b7c3 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:44.388602 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ca567584-3b39-4e38-bdb8-a7a9f644fbd2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.220373 Jul 05 10:13:44.389245 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 214/461] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:44 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:44.391870 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-cc9cb51d-55f4-432c-b8f1-104a3357b7c3 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1471 microversion: 2.1 time: 0.205215 Jul 05 10:13:44.396698 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 248/462] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:44 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1471 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:44.396844 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-eabaa267-9b9f-4b29-a6e2-089c4413ec3b tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:44.396844 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-eabaa267-9b9f-4b29-a6e2-089c4413ec3b tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:44.397685 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d68c193a-8d03-4d5f-ae9b-3873c69c6502 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:44.398784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d68c193a-8d03-4d5f-ae9b-3873c69c6502 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:44.400314 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eabaa267-9b9f-4b29-a6e2-089c4413ec3b tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:44.400957 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eabaa267-9b9f-4b29-a6e2-089c4413ec3b tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:44.401902 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eabaa267-9b9f-4b29-a6e2-089c4413ec3b tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:44.402334 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-eabaa267-9b9f-4b29-a6e2-089c4413ec3b tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:44.404575 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d68c193a-8d03-4d5f-ae9b-3873c69c6502 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:44.405015 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d68c193a-8d03-4d5f-ae9b-3873c69c6502 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:44.405762 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d68c193a-8d03-4d5f-ae9b-3873c69c6502 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:44.406189 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d68c193a-8d03-4d5f-ae9b-3873c69c6502 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:44.494267 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d68c193a-8d03-4d5f-ae9b-3873c69c6502 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:44.502381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d68c193a-8d03-4d5f-ae9b-3873c69c6502 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1186 microversion: 2.1 time: 0.108608 Jul 05 10:13:44.503233 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 249/463] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:44 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1186 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:44.608869 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-eabaa267-9b9f-4b29-a6e2-089c4413ec3b tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:44.616152 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-eabaa267-9b9f-4b29-a6e2-089c4413ec3b tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1319 microversion: 2.1 time: 0.225958 Jul 05 10:13:44.617379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 215/464] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:44 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1319 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:45.408196 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d06a58ee-8514-449a-b616-35546bd49f92 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:45.409285 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-48c65b33-dfb3-4a31-91c3-1c16cd36dba4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:45.410032 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-48c65b33-dfb3-4a31-91c3-1c16cd36dba4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:45.410271 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d06a58ee-8514-449a-b616-35546bd49f92 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:45.417677 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d06a58ee-8514-449a-b616-35546bd49f92 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:45.418452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d06a58ee-8514-449a-b616-35546bd49f92 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:45.419563 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d06a58ee-8514-449a-b616-35546bd49f92 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:45.420226 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d06a58ee-8514-449a-b616-35546bd49f92 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:45.424449 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-48c65b33-dfb3-4a31-91c3-1c16cd36dba4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:45.424919 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-48c65b33-dfb3-4a31-91c3-1c16cd36dba4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:45.427084 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-48c65b33-dfb3-4a31-91c3-1c16cd36dba4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:45.429731 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-48c65b33-dfb3-4a31-91c3-1c16cd36dba4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:45.718279 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-48c65b33-dfb3-4a31-91c3-1c16cd36dba4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:45.723162 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d06a58ee-8514-449a-b616-35546bd49f92 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:45.725784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-48c65b33-dfb3-4a31-91c3-1c16cd36dba4 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1471 microversion: 2.1 time: 0.320690 Jul 05 10:13:45.726963 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 216/465] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:45 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1471 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:45.731542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-b91e5807-3afb-4331-a5a2-89b6c3900974 req-e85550f2-e69c-459b-b7f3-4d6adb47f14a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "86fe0729-f79b-4fee-a79a-d004a32cfa2d", "tag": "b00d9c04-1d84-4211-8344-23f678f0b868"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:13:45.736584 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d06a58ee-8514-449a-b616-35546bd49f92 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1370 microversion: 2.1 time: 0.334103 Jul 05 10:13:45.737346 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 250/466] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:45 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1370 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:45.742473 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0f3838ef-c1ec-4ca4-ac4f-1a0699519281 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:45.743050 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0f3838ef-c1ec-4ca4-ac4f-1a0699519281 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:45.748074 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-b91e5807-3afb-4331-a5a2-89b6c3900974 req-e85550f2-e69c-459b-b7f3-4d6adb47f14a service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:45.748609 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-b91e5807-3afb-4331-a5a2-89b6c3900974 req-e85550f2-e69c-459b-b7f3-4d6adb47f14a service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:45.752234 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0f3838ef-c1ec-4ca4-ac4f-1a0699519281 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:45.753866 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0f3838ef-c1ec-4ca4-ac4f-1a0699519281 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:45.754890 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0f3838ef-c1ec-4ca4-ac4f-1a0699519281 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:45.755407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0f3838ef-c1ec-4ca4-ac4f-1a0699519281 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:45.762300 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-b91e5807-3afb-4331-a5a2-89b6c3900974 req-e85550f2-e69c-459b-b7f3-4d6adb47f14a service nova] Creating event network-changed:b00d9c04-1d84-4211-8344-23f678f0b868 for instance 86fe0729-f79b-4fee-a79a-d004a32cfa2d on hv-cinder-maste Jul 05 10:13:45.767275 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-b91e5807-3afb-4331-a5a2-89b6c3900974 req-e85550f2-e69c-459b-b7f3-4d6adb47f14a service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039179 Jul 05 10:13:45.768066 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 217/467] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:13:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:13:45.773343 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-4fcfb4b3-e7ee-4416-aebd-0beee75e816d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:45.774032 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-4fcfb4b3-e7ee-4416-aebd-0beee75e816d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:45.780174 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4fcfb4b3-e7ee-4416-aebd-0beee75e816d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:45.780676 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4fcfb4b3-e7ee-4416-aebd-0beee75e816d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:45.781446 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4fcfb4b3-e7ee-4416-aebd-0beee75e816d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:45.781886 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4fcfb4b3-e7ee-4416-aebd-0beee75e816d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:45.853484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0f3838ef-c1ec-4ca4-ac4f-1a0699519281 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:45.860897 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0f3838ef-c1ec-4ca4-ac4f-1a0699519281 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1194 microversion: 2.1 time: 0.122386 Jul 05 10:13:45.861460 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 251/468] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:45 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1194 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:46.013972 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-4fcfb4b3-e7ee-4416-aebd-0beee75e816d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:46.021378 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-4fcfb4b3-e7ee-4416-aebd-0beee75e816d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1319 microversion: 2.1 time: 0.252051 Jul 05 10:13:46.022030 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 218/469] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:45 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1319 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:46.742951 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2f0a9f81-c3e2-440e-b0a2-22382725f642 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:46.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2f0a9f81-c3e2-440e-b0a2-22382725f642 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:46.745698 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:46.751988 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f0a9f81-c3e2-440e-b0a2-22382725f642 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:46.752490 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f0a9f81-c3e2-440e-b0a2-22382725f642 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:46.753264 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f0a9f81-c3e2-440e-b0a2-22382725f642 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:46.753604 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d51f1613-abc6-4c34-8729-48aad729e2a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:46.754065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f0a9f81-c3e2-440e-b0a2-22382725f642 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:46.754213 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d51f1613-abc6-4c34-8729-48aad729e2a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:46.756914 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=4 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:46.762941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d51f1613-abc6-4c34-8729-48aad729e2a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:46.763506 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d51f1613-abc6-4c34-8729-48aad729e2a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:46.764155 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d51f1613-abc6-4c34-8729-48aad729e2a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:46.764618 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d51f1613-abc6-4c34-8729-48aad729e2a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:46.792259 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f0a9f81-c3e2-440e-b0a2-22382725f642 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:46.793156 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f0a9f81-c3e2-440e-b0a2-22382725f642 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:46.793815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.objects.instance [None req-2f0a9f81-c3e2-440e-b0a2-22382725f642 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lazy-loading 'fault' on Instance uuid 81676d9e-8ae6-4214-8763-1a143ffac94c {{(pid=117813) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:13:46.880452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2f0a9f81-c3e2-440e-b0a2-22382725f642 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa8cee3ae343415aae94ce2592a04dd3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9bf4c2e6ff4347f5b5e46efe5d431dc9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:46.889051 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2f0a9f81-c3e2-440e-b0a2-22382725f642 tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 200 len: 1305 microversion: 2.1 time: 0.150056 Jul 05 10:13:46.890146 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 252/470] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:46 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 1305 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:46.894856 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-758b743e-48ed-4ada-88b7-75bbcf5359be tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:46.895652 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-758b743e-48ed-4ada-88b7-75bbcf5359be tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:46.902927 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-758b743e-48ed-4ada-88b7-75bbcf5359be tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:46.903374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-758b743e-48ed-4ada-88b7-75bbcf5359be tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:46.904138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-758b743e-48ed-4ada-88b7-75bbcf5359be tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:46.904598 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-758b743e-48ed-4ada-88b7-75bbcf5359be tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:47.040945 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-758b743e-48ed-4ada-88b7-75bbcf5359be tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:47.048410 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-758b743e-48ed-4ada-88b7-75bbcf5359be tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1204 microversion: 2.1 time: 0.157000 Jul 05 10:13:47.048984 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 253/471] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:46 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1204 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:47.055230 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-28d24027-7f34-405c-80cd-ca78f704d1f9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:47.056096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-28d24027-7f34-405c-80cd-ca78f704d1f9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:47.061544 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-28d24027-7f34-405c-80cd-ca78f704d1f9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:47.062362 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-28d24027-7f34-405c-80cd-ca78f704d1f9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:47.062742 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-28d24027-7f34-405c-80cd-ca78f704d1f9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:47.063163 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-28d24027-7f34-405c-80cd-ca78f704d1f9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:47.131356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=14 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:47.251574 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d51f1613-abc6-4c34-8729-48aad729e2a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:47.260132 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d51f1613-abc6-4c34-8729-48aad729e2a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1564 microversion: 2.1 time: 0.510173 Jul 05 10:13:47.260747 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 219/472] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:46 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1564 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:47.277235 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-bc5327c5-df00-4e2e-ae72-9c1a191ecc1f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:47.278127 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bc5327c5-df00-4e2e-ae72-9c1a191ecc1f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:47.285508 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bc5327c5-df00-4e2e-ae72-9c1a191ecc1f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:47.285866 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bc5327c5-df00-4e2e-ae72-9c1a191ecc1f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:47.287008 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bc5327c5-df00-4e2e-ae72-9c1a191ecc1f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:47.287557 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bc5327c5-df00-4e2e-ae72-9c1a191ecc1f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:47.310104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=100 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:47.463762 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-28d24027-7f34-405c-80cd-ca78f704d1f9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:47.471403 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-28d24027-7f34-405c-80cd-ca78f704d1f9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1319 microversion: 2.1 time: 0.421351 Jul 05 10:13:47.472182 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 254/473] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:47 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1319 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:47.483297 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-bc5327c5-df00-4e2e-ae72-9c1a191ecc1f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.210182 Jul 05 10:13:47.483888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 220/474] 10.222.0.20 () {58 vars in 1274 bytes} [Tue Jul 5 10:13:47 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-interface => generated 285 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:13:47.792637 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-f01d5093-a80e-4e2e-a539-fbd528d269fb req-5f6b89a9-b843-4a1f-b889-5dfbc7ede992 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "81676d9e-8ae6-4214-8763-1a143ffac94c", "name": "network-vif-deleted", "tag": "c4b5a44e-2a0f-4e9f-bcaa-a8472a69f6a5"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:13:47.801050 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-f01d5093-a80e-4e2e-a539-fbd528d269fb req-5f6b89a9-b843-4a1f-b889-5dfbc7ede992 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:47.801489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-f01d5093-a80e-4e2e-a539-fbd528d269fb req-5f6b89a9-b843-4a1f-b889-5dfbc7ede992 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:47.813617 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-f01d5093-a80e-4e2e-a539-fbd528d269fb req-5f6b89a9-b843-4a1f-b889-5dfbc7ede992 service nova] Creating event network-vif-deleted:c4b5a44e-2a0f-4e9f-bcaa-a8472a69f6a5 for instance 81676d9e-8ae6-4214-8763-1a143ffac94c on hv-cinder-maste Jul 05 10:13:47.818728 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-f01d5093-a80e-4e2e-a539-fbd528d269fb req-5f6b89a9-b843-4a1f-b889-5dfbc7ede992 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031158 Jul 05 10:13:47.819419 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 255/475] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:13:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:13:47.908551 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-70167be0-a3a3-4221-b3da-6657d69cd18a tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:47.909915 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-70167be0-a3a3-4221-b3da-6657d69cd18a tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] [instance: 81676d9e-8ae6-4214-8763-1a143ffac94c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:47.921186 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-70167be0-a3a3-4221-b3da-6657d69cd18a tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:47.921883 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-70167be0-a3a3-4221-b3da-6657d69cd18a tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:47.923465 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-70167be0-a3a3-4221-b3da-6657d69cd18a tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:47.924727 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-70167be0-a3a3-4221-b3da-6657d69cd18a tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:47.945179 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-70167be0-a3a3-4221-b3da-6657d69cd18a tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] HTTP exception thrown: Instance 81676d9e-8ae6-4214-8763-1a143ffac94c could not be found. Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-70167be0-a3a3-4221-b3da-6657d69cd18a tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Request method failure captured: Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c HTTP/1.1 Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA6R5hs_FDVAQQx5Iu-7K64SIkem_NqY6UI0dysezF0lfy9vdI78qh4ONcHNXrlKXyaZ4euP4sE3QY-quERxEfg8n2TY6ydLOJPiBYH4AVcikFYj8RMFwPdM3flTWCDHVHr6c4O5wOpJ9KspkGULB1URPVSUOnH2CmZlmd0tiVU32_k Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: 9bf4c2e6ff4347f5b5e46efe5d431dc9 Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-VolumesNegativeTest-649096789 Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:13:47.947262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:13:47.948737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9bf4c2e6ff4347f5b5e46efe5d431dc9"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9bf4c2e6ff4347f5b5e46efe5d431dc9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9bf4c2e6ff4347f5b5e46efe5d431dc9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9bf4c2e6ff4347f5b5e46efe5d431dc9"}]}] Jul 05 10:13:47.948737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-VolumesNegativeTest-649096789 Jul 05 10:13:47.948737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: 9bf4c2e6ff4347f5b5e46efe5d431dc9 Jul 05 10:13:47.948737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-VolumesNegativeTest-649096789 Jul 05 10:13:47.948737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-VolumesNegativeTest-649096789-project Jul 05 10:13:47.948737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:13:47.948737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:13:47.948737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: aa8cee3ae343415aae94ce2592a04dd3 Jul 05 10:13:47.948737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-VolumesNegativeTest-649096789-project Jul 05 10:13:47.948737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:13:47.948737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance 81676d9e-8ae6-4214-8763-1a143ffac94c could not be found. Jul 05 10:13:47.948737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': '81676d9e-8ae6-4214-8763-1a143ffac94c'} Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 81676d9e-8ae6-4214-8763-1a143ffac94c could not be found. Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:13:47.949483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 81676d9e-8ae6-4214-8763-1a143ffac94c could not be found. Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 81676d9e-8ae6-4214-8763-1a143ffac94c could not be found. Jul 05 10:13:47.950738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:13:47.953321 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-70167be0-a3a3-4221-b3da-6657d69cd18a tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] Returning 404 to user: Instance 81676d9e-8ae6-4214-8763-1a143ffac94c could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:13:47.954368 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-70167be0-a3a3-4221-b3da-6657d69cd18a tempest-VolumesNegativeTest-649096789 tempest-VolumesNegativeTest-649096789-project] 10.222.0.20 "GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c" status: 404 len: 111 microversion: 2.1 time: 0.051548 Jul 05 10:13:47.958430 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 221/476] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:47 2022] GET /compute/v2.1/servers/81676d9e-8ae6-4214-8763-1a143ffac94c => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:13:48.066013 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a8367ecd-100e-44c5-ba5a-8ef266ed4261 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:48.066702 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a8367ecd-100e-44c5-ba5a-8ef266ed4261 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:48.084638 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a8367ecd-100e-44c5-ba5a-8ef266ed4261 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:48.085091 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a8367ecd-100e-44c5-ba5a-8ef266ed4261 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:48.093067 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a8367ecd-100e-44c5-ba5a-8ef266ed4261 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:48.093686 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a8367ecd-100e-44c5-ba5a-8ef266ed4261 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:48.335278 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a8367ecd-100e-44c5-ba5a-8ef266ed4261 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:48.361430 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a8367ecd-100e-44c5-ba5a-8ef266ed4261 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.284510 Jul 05 10:13:48.364837 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 256/477] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:48 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:48.489418 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d405186c-a4bb-4dde-a83c-af800f375848 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:48.490040 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d405186c-a4bb-4dde-a83c-af800f375848 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:48.496364 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d405186c-a4bb-4dde-a83c-af800f375848 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:48.507149 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d405186c-a4bb-4dde-a83c-af800f375848 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:48.508713 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d405186c-a4bb-4dde-a83c-af800f375848 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:48.509311 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d405186c-a4bb-4dde-a83c-af800f375848 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:48.723936 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d405186c-a4bb-4dde-a83c-af800f375848 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:48.743036 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d405186c-a4bb-4dde-a83c-af800f375848 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1319 microversion: 2.1 time: 0.250517 Jul 05 10:13:48.746870 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 222/478] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:48 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1319 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:48.997793 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-7cabd9d3-9604-4da4-9572-6df748992a92 req-478d5ac8-4a45-466d-8639-5f2140603188 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "12e7fd00-8f3e-4fc4-882b-6ec193de0bfd", "tag": "e27d9ee6-1685-4aec-bb2d-ae3b3206c610"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:13:49.009262 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-7cabd9d3-9604-4da4-9572-6df748992a92 req-478d5ac8-4a45-466d-8639-5f2140603188 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:49.009493 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-7cabd9d3-9604-4da4-9572-6df748992a92 req-478d5ac8-4a45-466d-8639-5f2140603188 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:49.024796 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-7cabd9d3-9604-4da4-9572-6df748992a92 req-478d5ac8-4a45-466d-8639-5f2140603188 service nova] Creating event network-changed:e27d9ee6-1685-4aec-bb2d-ae3b3206c610 for instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd on hv-cinder-maste Jul 05 10:13:49.031466 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-7cabd9d3-9604-4da4-9572-6df748992a92 req-478d5ac8-4a45-466d-8639-5f2140603188 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043576 Jul 05 10:13:49.032523 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 257/479] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:13:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:13:49.387516 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-bfb53dcd-0916-4284-9f6b-95d14c405dc8 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:49.388644 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bfb53dcd-0916-4284-9f6b-95d14c405dc8 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:49.401383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bfb53dcd-0916-4284-9f6b-95d14c405dc8 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:49.402571 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bfb53dcd-0916-4284-9f6b-95d14c405dc8 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:49.403352 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bfb53dcd-0916-4284-9f6b-95d14c405dc8 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:49.403869 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bfb53dcd-0916-4284-9f6b-95d14c405dc8 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:49.621948 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-bfb53dcd-0916-4284-9f6b-95d14c405dc8 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:49.635116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-bfb53dcd-0916-4284-9f6b-95d14c405dc8 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.251343 Jul 05 10:13:49.636600 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 223/480] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:49 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:49.764632 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-62267da7-0b94-4fbd-9648-39a9a37b7b4c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:49.765842 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-62267da7-0b94-4fbd-9648-39a9a37b7b4c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:49.774320 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-62267da7-0b94-4fbd-9648-39a9a37b7b4c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:49.774831 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-62267da7-0b94-4fbd-9648-39a9a37b7b4c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:49.775667 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-62267da7-0b94-4fbd-9648-39a9a37b7b4c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:49.776169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-62267da7-0b94-4fbd-9648-39a9a37b7b4c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:49.976185 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-62267da7-0b94-4fbd-9648-39a9a37b7b4c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:49.985003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-62267da7-0b94-4fbd-9648-39a9a37b7b4c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1319 microversion: 2.1 time: 0.224862 Jul 05 10:13:49.985802 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 258/481] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:49 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1319 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:50.651217 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ea5c438f-fbe1-4c22-b874-283ae287a2ac tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:50.652409 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ea5c438f-fbe1-4c22-b874-283ae287a2ac tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:50.667706 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ea5c438f-fbe1-4c22-b874-283ae287a2ac tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:50.668544 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ea5c438f-fbe1-4c22-b874-283ae287a2ac tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:50.669566 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ea5c438f-fbe1-4c22-b874-283ae287a2ac tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:50.670201 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ea5c438f-fbe1-4c22-b874-283ae287a2ac tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:51.002074 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ae2c9f75-b3e0-49cd-8f23-b465bb41ce12 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:51.002698 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ae2c9f75-b3e0-49cd-8f23-b465bb41ce12 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:51.015905 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae2c9f75-b3e0-49cd-8f23-b465bb41ce12 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:51.016363 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae2c9f75-b3e0-49cd-8f23-b465bb41ce12 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:51.022138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae2c9f75-b3e0-49cd-8f23-b465bb41ce12 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:51.022602 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae2c9f75-b3e0-49cd-8f23-b465bb41ce12 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:51.720748 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ea5c438f-fbe1-4c22-b874-283ae287a2ac tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:51.736205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ea5c438f-fbe1-4c22-b874-283ae287a2ac tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 1.086226 Jul 05 10:13:51.736910 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 224/482] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:50 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 1090 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:51.768691 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ae2c9f75-b3e0-49cd-8f23-b465bb41ce12 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:51.782523 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ae2c9f75-b3e0-49cd-8f23-b465bb41ce12 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1319 microversion: 2.1 time: 0.784182 Jul 05 10:13:51.783422 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 259/483] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:50 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1319 bytes in 786 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:52.353562 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-8ea14b31-b359-40b9-b588-f4d7bd72019c req-5b88c631-8d38-4538-a5d9-5370a836f1b7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4684ec3d-0407-461f-8a19-d6d8f99ca49b", "tag": "1a0fe351-4265-449d-8b7d-508a71c44b30"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:13:52.379072 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-8ea14b31-b359-40b9-b588-f4d7bd72019c req-5b88c631-8d38-4538-a5d9-5370a836f1b7 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:52.379537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-8ea14b31-b359-40b9-b588-f4d7bd72019c req-5b88c631-8d38-4538-a5d9-5370a836f1b7 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:52.397141 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-8ea14b31-b359-40b9-b588-f4d7bd72019c req-5b88c631-8d38-4538-a5d9-5370a836f1b7 service nova] Creating event network-changed:1a0fe351-4265-449d-8b7d-508a71c44b30 for instance 4684ec3d-0407-461f-8a19-d6d8f99ca49b on hv-cinder-maste Jul 05 10:13:52.403473 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-8ea14b31-b359-40b9-b588-f4d7bd72019c req-5b88c631-8d38-4538-a5d9-5370a836f1b7 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070924 Jul 05 10:13:52.411636 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 225/484] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:13:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:13:52.753733 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-59705e83-117b-4c7f-b81b-550b82a59209 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:52.755077 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-59705e83-117b-4c7f-b81b-550b82a59209 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:52.763078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-59705e83-117b-4c7f-b81b-550b82a59209 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:52.764108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-59705e83-117b-4c7f-b81b-550b82a59209 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:52.765093 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-59705e83-117b-4c7f-b81b-550b82a59209 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:52.765681 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-59705e83-117b-4c7f-b81b-550b82a59209 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:52.801189 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-835099c1-7ac3-49a5-8107-97e768dbe4dc tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:52.803378 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-835099c1-7ac3-49a5-8107-97e768dbe4dc tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:52.815535 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-835099c1-7ac3-49a5-8107-97e768dbe4dc tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:52.815995 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-835099c1-7ac3-49a5-8107-97e768dbe4dc tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:52.817129 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-835099c1-7ac3-49a5-8107-97e768dbe4dc tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:52.817574 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-835099c1-7ac3-49a5-8107-97e768dbe4dc tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:53.067660 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-835099c1-7ac3-49a5-8107-97e768dbe4dc tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:53.071866 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-59705e83-117b-4c7f-b81b-550b82a59209 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:53.075589 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-835099c1-7ac3-49a5-8107-97e768dbe4dc tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1319 microversion: 2.1 time: 0.279119 Jul 05 10:13:53.076153 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 226/485] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:52 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1319 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:53.079452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-59705e83-117b-4c7f-b81b-550b82a59209 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.329669 Jul 05 10:13:53.080012 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 260/486] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:52 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:53.453528 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-7c4998d8-df79-44db-bad0-799d1ccb648d req-8f55e9b6-d649-489c-a008-e5be2f60eb0a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "86fe0729-f79b-4fee-a79a-d004a32cfa2d", "name": "network-vif-plugged", "status": "completed", "tag": "b00d9c04-1d84-4211-8344-23f678f0b868"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:13:53.461893 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-7c4998d8-df79-44db-bad0-799d1ccb648d req-8f55e9b6-d649-489c-a008-e5be2f60eb0a service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:53.462337 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-7c4998d8-df79-44db-bad0-799d1ccb648d req-8f55e9b6-d649-489c-a008-e5be2f60eb0a service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:53.473189 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-7c4998d8-df79-44db-bad0-799d1ccb648d req-8f55e9b6-d649-489c-a008-e5be2f60eb0a service nova] Creating event network-vif-plugged:b00d9c04-1d84-4211-8344-23f678f0b868 for instance 86fe0729-f79b-4fee-a79a-d004a32cfa2d on hv-cinder-maste Jul 05 10:13:53.478336 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-7c4998d8-df79-44db-bad0-799d1ccb648d req-8f55e9b6-d649-489c-a008-e5be2f60eb0a service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028728 Jul 05 10:13:53.478914 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 227/487] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:13:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:13:54.092787 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-455b606e-75ac-4ec1-ba8c-96167b24414a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:54.093542 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-455b606e-75ac-4ec1-ba8c-96167b24414a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:54.097036 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-68b7c6b9-def5-4995-bc5b-237fda55740e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:54.098571 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-68b7c6b9-def5-4995-bc5b-237fda55740e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:54.099461 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-455b606e-75ac-4ec1-ba8c-96167b24414a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:54.099964 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-455b606e-75ac-4ec1-ba8c-96167b24414a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:54.100990 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-455b606e-75ac-4ec1-ba8c-96167b24414a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:54.101651 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-455b606e-75ac-4ec1-ba8c-96167b24414a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:54.105148 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-68b7c6b9-def5-4995-bc5b-237fda55740e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:54.105605 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-68b7c6b9-def5-4995-bc5b-237fda55740e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:54.106421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-68b7c6b9-def5-4995-bc5b-237fda55740e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:54.106861 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-68b7c6b9-def5-4995-bc5b-237fda55740e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:54.311544 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-68b7c6b9-def5-4995-bc5b-237fda55740e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:54.313705 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-455b606e-75ac-4ec1-ba8c-96167b24414a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:54.339105 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-455b606e-75ac-4ec1-ba8c-96167b24414a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1319 microversion: 2.1 time: 0.249778 Jul 05 10:13:54.345344 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-68b7c6b9-def5-4995-bc5b-237fda55740e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.252892 Jul 05 10:13:54.346308 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 228/488] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:54 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:13:54.347466 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 261/489] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:54 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1319 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:55.363061 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ce8cff72-1ecd-464e-88d5-0e697a2ca170 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:55.368722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2a8c281f-8f63-4239-be6c-1d818dc29985 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:55.369328 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2a8c281f-8f63-4239-be6c-1d818dc29985 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:55.372607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ce8cff72-1ecd-464e-88d5-0e697a2ca170 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:55.375887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2a8c281f-8f63-4239-be6c-1d818dc29985 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:55.376506 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2a8c281f-8f63-4239-be6c-1d818dc29985 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:55.377378 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2a8c281f-8f63-4239-be6c-1d818dc29985 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:55.377881 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2a8c281f-8f63-4239-be6c-1d818dc29985 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:55.379189 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ce8cff72-1ecd-464e-88d5-0e697a2ca170 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:55.379807 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ce8cff72-1ecd-464e-88d5-0e697a2ca170 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:55.380629 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ce8cff72-1ecd-464e-88d5-0e697a2ca170 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:55.381093 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ce8cff72-1ecd-464e-88d5-0e697a2ca170 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:55.654858 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ce8cff72-1ecd-464e-88d5-0e697a2ca170 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:55.665335 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2a8c281f-8f63-4239-be6c-1d818dc29985 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:55.670087 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ce8cff72-1ecd-464e-88d5-0e697a2ca170 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.310710 Jul 05 10:13:55.670907 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 229/490] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:55 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:55.683037 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2a8c281f-8f63-4239-be6c-1d818dc29985 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1319 microversion: 2.1 time: 0.318507 Jul 05 10:13:55.683947 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 262/491] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:55 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1319 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:56.688019 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5fd93a0c-6ca0-4b8c-b5ed-af591fe83106 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:56.690029 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5fd93a0c-6ca0-4b8c-b5ed-af591fe83106 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:56.700530 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5fd93a0c-6ca0-4b8c-b5ed-af591fe83106 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:56.701410 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5fd93a0c-6ca0-4b8c-b5ed-af591fe83106 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:56.701658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-38992b7a-c6de-4925-ab76-bb03e3b75a2f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:56.702163 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-38992b7a-c6de-4925-ab76-bb03e3b75a2f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:56.703418 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5fd93a0c-6ca0-4b8c-b5ed-af591fe83106 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:56.704076 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5fd93a0c-6ca0-4b8c-b5ed-af591fe83106 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:56.713370 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-38992b7a-c6de-4925-ab76-bb03e3b75a2f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:56.713836 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-38992b7a-c6de-4925-ab76-bb03e3b75a2f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:56.714633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-38992b7a-c6de-4925-ab76-bb03e3b75a2f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:56.715095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-38992b7a-c6de-4925-ab76-bb03e3b75a2f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:57.038759 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5fd93a0c-6ca0-4b8c-b5ed-af591fe83106 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:57.055724 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5fd93a0c-6ca0-4b8c-b5ed-af591fe83106 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.371283 Jul 05 10:13:57.055834 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 230/492] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:56 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:57.099055 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-38992b7a-c6de-4925-ab76-bb03e3b75a2f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:57.111589 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-38992b7a-c6de-4925-ab76-bb03e3b75a2f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1507 microversion: 2.1 time: 0.414734 Jul 05 10:13:57.112746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 263/493] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:56 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1507 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:58.072011 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b99798af-ffed-426c-99f0-66c546ec8b9d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:58.072915 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b99798af-ffed-426c-99f0-66c546ec8b9d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:58.089766 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b99798af-ffed-426c-99f0-66c546ec8b9d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:58.090575 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b99798af-ffed-426c-99f0-66c546ec8b9d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:58.092966 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b99798af-ffed-426c-99f0-66c546ec8b9d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:58.093416 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b99798af-ffed-426c-99f0-66c546ec8b9d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:58.360115 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b99798af-ffed-426c-99f0-66c546ec8b9d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:58.367849 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b99798af-ffed-426c-99f0-66c546ec8b9d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.300711 Jul 05 10:13:58.368664 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 231/494] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:58 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:13:59.384859 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-4a02539a-5ba7-4730-b04d-5653417c4a64 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:13:59.385470 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-4a02539a-5ba7-4730-b04d-5653417c4a64 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:13:59.391815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4a02539a-5ba7-4730-b04d-5653417c4a64 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:59.392258 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4a02539a-5ba7-4730-b04d-5653417c4a64 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:59.393256 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4a02539a-5ba7-4730-b04d-5653417c4a64 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:13:59.393692 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4a02539a-5ba7-4730-b04d-5653417c4a64 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:13:59.633870 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4a02539a-5ba7-4730-b04d-5653417c4a64 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:13:59.640732 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=100 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:13:59.644027 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-4a02539a-5ba7-4730-b04d-5653417c4a64 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.263025 Jul 05 10:13:59.644726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 264/495] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:13:59 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:00.663075 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f35001d5-3b55-4afc-be02-c2b390a5e5a4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:00.663781 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f35001d5-3b55-4afc-be02-c2b390a5e5a4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:00.670922 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f35001d5-3b55-4afc-be02-c2b390a5e5a4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:00.671516 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f35001d5-3b55-4afc-be02-c2b390a5e5a4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:00.672494 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f35001d5-3b55-4afc-be02-c2b390a5e5a4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:00.673228 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f35001d5-3b55-4afc-be02-c2b390a5e5a4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:00.910007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f35001d5-3b55-4afc-be02-c2b390a5e5a4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:00.938709 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f35001d5-3b55-4afc-be02-c2b390a5e5a4 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.280229 Jul 05 10:14:00.939804 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 232/496] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:00 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:01.956718 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c5ed340a-89e7-4393-96e2-21831aca0e6e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:01.957998 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c5ed340a-89e7-4393-96e2-21831aca0e6e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:01.966032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c5ed340a-89e7-4393-96e2-21831aca0e6e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:01.966697 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c5ed340a-89e7-4393-96e2-21831aca0e6e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:01.967686 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c5ed340a-89e7-4393-96e2-21831aca0e6e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:01.968860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c5ed340a-89e7-4393-96e2-21831aca0e6e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:02.172994 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c5ed340a-89e7-4393-96e2-21831aca0e6e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:02.182400 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c5ed340a-89e7-4393-96e2-21831aca0e6e tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.229564 Jul 05 10:14:02.183194 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 265/497] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:01 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:03.199622 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f61a81a5-6250-4ce7-843d-cd36095d6d24 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:03.203248 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f61a81a5-6250-4ce7-843d-cd36095d6d24 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:03.213302 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f61a81a5-6250-4ce7-843d-cd36095d6d24 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:03.214226 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f61a81a5-6250-4ce7-843d-cd36095d6d24 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:03.217243 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f61a81a5-6250-4ce7-843d-cd36095d6d24 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:03.218151 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f61a81a5-6250-4ce7-843d-cd36095d6d24 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:03.452613 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f61a81a5-6250-4ce7-843d-cd36095d6d24 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:03.465686 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f61a81a5-6250-4ce7-843d-cd36095d6d24 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.270872 Jul 05 10:14:03.477501 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 233/498] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:03 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:04.485105 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-56a2128a-2a15-4230-afb4-fb7ed7cd0fab tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:04.485722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-56a2128a-2a15-4230-afb4-fb7ed7cd0fab tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:04.507891 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-56a2128a-2a15-4230-afb4-fb7ed7cd0fab tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:04.511565 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-56a2128a-2a15-4230-afb4-fb7ed7cd0fab tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:04.512786 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-56a2128a-2a15-4230-afb4-fb7ed7cd0fab tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:04.513330 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-56a2128a-2a15-4230-afb4-fb7ed7cd0fab tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:04.700739 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-56a2128a-2a15-4230-afb4-fb7ed7cd0fab tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:04.708889 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-56a2128a-2a15-4230-afb4-fb7ed7cd0fab tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.227710 Jul 05 10:14:04.709454 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 266/499] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:04 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:05.726933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-094fab26-525d-485d-a98f-eec4ea523d22 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:05.727552 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-094fab26-525d-485d-a98f-eec4ea523d22 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:05.733622 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-094fab26-525d-485d-a98f-eec4ea523d22 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:05.734558 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-094fab26-525d-485d-a98f-eec4ea523d22 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:05.734835 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-094fab26-525d-485d-a98f-eec4ea523d22 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:05.735754 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-094fab26-525d-485d-a98f-eec4ea523d22 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:05.900796 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-094fab26-525d-485d-a98f-eec4ea523d22 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:05.908585 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-094fab26-525d-485d-a98f-eec4ea523d22 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.186112 Jul 05 10:14:05.924547 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 234/500] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:05 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:14:06.938767 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e941b95c-ff55-4638-9ea5-789f0d16f3d0 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:06.940317 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e941b95c-ff55-4638-9ea5-789f0d16f3d0 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:06.951646 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=73,nova_api:UPDATE=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:06.953419 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e941b95c-ff55-4638-9ea5-789f0d16f3d0 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:06.976974 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e941b95c-ff55-4638-9ea5-789f0d16f3d0 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:06.979667 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e941b95c-ff55-4638-9ea5-789f0d16f3d0 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:06.980311 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e941b95c-ff55-4638-9ea5-789f0d16f3d0 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:07.277183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e941b95c-ff55-4638-9ea5-789f0d16f3d0 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:07.286276 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e941b95c-ff55-4638-9ea5-789f0d16f3d0 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.359728 Jul 05 10:14:07.287170 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 267/501] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:06 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:08.303600 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e21bbcbf-be0f-4e80-86c4-70967608dd9b tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:08.311445 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e21bbcbf-be0f-4e80-86c4-70967608dd9b tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:08.322039 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e21bbcbf-be0f-4e80-86c4-70967608dd9b tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:08.322756 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e21bbcbf-be0f-4e80-86c4-70967608dd9b tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:08.325788 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e21bbcbf-be0f-4e80-86c4-70967608dd9b tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:08.326460 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e21bbcbf-be0f-4e80-86c4-70967608dd9b tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:08.492071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e21bbcbf-be0f-4e80-86c4-70967608dd9b tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:08.502935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e21bbcbf-be0f-4e80-86c4-70967608dd9b tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.203369 Jul 05 10:14:08.503804 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 235/502] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:08 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:09.531427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-204dc114-42a0-49c7-b364-cec1cc3114cd tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:09.531932 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-204dc114-42a0-49c7-b364-cec1cc3114cd tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:09.546818 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-204dc114-42a0-49c7-b364-cec1cc3114cd tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:09.553539 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-204dc114-42a0-49c7-b364-cec1cc3114cd tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:09.554587 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-204dc114-42a0-49c7-b364-cec1cc3114cd tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:09.556705 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-204dc114-42a0-49c7-b364-cec1cc3114cd tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:09.829091 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-204dc114-42a0-49c7-b364-cec1cc3114cd tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:09.851371 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-204dc114-42a0-49c7-b364-cec1cc3114cd tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.333005 Jul 05 10:14:09.852938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 268/503] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:09 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:10.871967 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-537fe0cc-a54f-4137-8172-30a48aa827bb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:10.872648 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-537fe0cc-a54f-4137-8172-30a48aa827bb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:10.881717 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=53 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:10.883405 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-537fe0cc-a54f-4137-8172-30a48aa827bb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:10.884040 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-537fe0cc-a54f-4137-8172-30a48aa827bb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:10.885139 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-537fe0cc-a54f-4137-8172-30a48aa827bb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:10.896013 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-537fe0cc-a54f-4137-8172-30a48aa827bb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:11.113009 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-537fe0cc-a54f-4137-8172-30a48aa827bb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:11.123155 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-537fe0cc-a54f-4137-8172-30a48aa827bb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.257905 Jul 05 10:14:11.135044 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 236/504] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:10 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:11.882829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-d54dced3-7456-4c03-831f-047e63fd52bd req-d8503292-dea7-4344-bbab-d1cc73a0e519 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4684ec3d-0407-461f-8a19-d6d8f99ca49b", "name": "network-vif-plugged", "status": "completed", "tag": "1a0fe351-4265-449d-8b7d-508a71c44b30"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:14:11.897682 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-d54dced3-7456-4c03-831f-047e63fd52bd req-d8503292-dea7-4344-bbab-d1cc73a0e519 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:11.898394 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-d54dced3-7456-4c03-831f-047e63fd52bd req-d8503292-dea7-4344-bbab-d1cc73a0e519 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:11.915489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-d54dced3-7456-4c03-831f-047e63fd52bd req-d8503292-dea7-4344-bbab-d1cc73a0e519 service nova] Creating event network-vif-plugged:1a0fe351-4265-449d-8b7d-508a71c44b30 for instance 4684ec3d-0407-461f-8a19-d6d8f99ca49b on hv-cinder-maste Jul 05 10:14:11.923340 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-d54dced3-7456-4c03-831f-047e63fd52bd req-d8503292-dea7-4344-bbab-d1cc73a0e519 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044098 Jul 05 10:14:11.924671 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 269/505] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:14:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:14:12.139720 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5217cdf9-ea65-436f-8885-9d81f10d2efe tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:12.140680 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5217cdf9-ea65-436f-8885-9d81f10d2efe tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:12.147888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5217cdf9-ea65-436f-8885-9d81f10d2efe tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:12.148929 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5217cdf9-ea65-436f-8885-9d81f10d2efe tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:12.150617 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5217cdf9-ea65-436f-8885-9d81f10d2efe tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:12.151359 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5217cdf9-ea65-436f-8885-9d81f10d2efe tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:12.310929 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5217cdf9-ea65-436f-8885-9d81f10d2efe tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:12.319372 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5217cdf9-ea65-436f-8885-9d81f10d2efe tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.183394 Jul 05 10:14:12.319970 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 237/506] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:12 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:13.339611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-196031eb-20d2-4b92-a809-71268eefd7aa tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:13.340304 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-196031eb-20d2-4b92-a809-71268eefd7aa tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:13.346768 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-196031eb-20d2-4b92-a809-71268eefd7aa tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:13.347237 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-196031eb-20d2-4b92-a809-71268eefd7aa tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:13.348190 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-196031eb-20d2-4b92-a809-71268eefd7aa tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:13.348669 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-196031eb-20d2-4b92-a809-71268eefd7aa tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:13.496762 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-196031eb-20d2-4b92-a809-71268eefd7aa tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:13.505025 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-196031eb-20d2-4b92-a809-71268eefd7aa tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.171562 Jul 05 10:14:13.505592 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 270/507] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:13 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:14.521474 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-94f5c24f-eeb0-4ef8-9d25-3a536f0af5ee tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:14.522176 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-94f5c24f-eeb0-4ef8-9d25-3a536f0af5ee tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:14.529511 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-94f5c24f-eeb0-4ef8-9d25-3a536f0af5ee tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:14.530197 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-94f5c24f-eeb0-4ef8-9d25-3a536f0af5ee tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:14.531096 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-94f5c24f-eeb0-4ef8-9d25-3a536f0af5ee tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:14.531539 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-94f5c24f-eeb0-4ef8-9d25-3a536f0af5ee tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:14.693451 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-94f5c24f-eeb0-4ef8-9d25-3a536f0af5ee tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:14.705086 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-94f5c24f-eeb0-4ef8-9d25-3a536f0af5ee tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1259 microversion: 2.1 time: 0.187273 Jul 05 10:14:14.705914 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 238/508] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:14 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1259 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:15.724691 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-cb89f376-d329-491a-a500-ee7d55980d2a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:15.725220 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-cb89f376-d329-491a-a500-ee7d55980d2a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:15.732969 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cb89f376-d329-491a-a500-ee7d55980d2a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:15.733582 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cb89f376-d329-491a-a500-ee7d55980d2a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:15.734535 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cb89f376-d329-491a-a500-ee7d55980d2a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:15.735064 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cb89f376-d329-491a-a500-ee7d55980d2a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:15.798321 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-08a92186-b0aa-4eed-91a1-4da5988a7f44 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Action: 'create', calling method: .version_select at 0x7feb89794700>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-934834031", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBO4jJYvBvUWOoHgyoXGcWK3ImB9t59m0TzjrVhvMbY5aLGyF4AyZvqIFPDGBdQtBeuK6dgd8aMB44c6VYVY7+0SGWMnxXxcIa0ZryfQL561tA6990MkM94nxqH0S6Xe3Pg=="}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:14:15.815307 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-08a92186-b0aa-4eed-91a1-4da5988a7f44 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Getting quotas for project d71ebc02c6094919b3054d7da84aadf0. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:14:15.819201 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-08a92186-b0aa-4eed-91a1-4da5988a7f44 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Getting quotas for user f0351b9ab5ac4fbaad38df073518412a and project d71ebc02c6094919b3054d7da84aadf0. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:14:15.843613 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-08a92186-b0aa-4eed-91a1-4da5988a7f44 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.049012 Jul 05 10:14:15.844314 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 239/509] 10.222.0.20 () {60 vars in 1158 bytes} [Tue Jul 5 10:14:15 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:14:15.892279 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-179699683", "imageRef": "9854fd19-f34c-4ada-b3cb-25f1b62d0766", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-934834031", "networks": [{"uuid": "71cc70a2-9310-4a30-a617-ac7068d14bb6"}]}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:14:15.946591 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.network.neutron [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] validate_networks() for [('71cc70a2-9310-4a30-a617-ac7068d14bb6', None, None, None, None, None)] {{(pid=117812) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:14:15.949292 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-cb89f376-d329-491a-a500-ee7d55980d2a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:15.957103 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-cb89f376-d329-491a-a500-ee7d55980d2a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1443 microversion: 2.1 time: 0.238003 Jul 05 10:14:15.957661 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 271/510] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:15 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1443 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:15.974346 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-31b4609f-d095-42b7-a46e-465aca8d7f57 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:15.974864 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-31b4609f-d095-42b7-a46e-465aca8d7f57 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:15.981029 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-31b4609f-d095-42b7-a46e-465aca8d7f57 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:15.981473 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-31b4609f-d095-42b7-a46e-465aca8d7f57 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:15.982468 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-31b4609f-d095-42b7-a46e-465aca8d7f57 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:15.982698 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-31b4609f-d095-42b7-a46e-465aca8d7f57 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:16.269786 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-31b4609f-d095-42b7-a46e-465aca8d7f57 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:16.277490 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-31b4609f-d095-42b7-a46e-465aca8d7f57 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1443 microversion: 2.1 time: 0.307042 Jul 05 10:14:16.278061 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 272/511] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:15 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1443 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:14:16.528353 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Flavor limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:14:16.529701 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Image limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:14:16.530356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Flavor pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:14:16.531517 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Image pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:14:16.540496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:14:16.549259 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:16.549704 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:16.550490 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:16.550922 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:16.567375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Getting quotas for project d71ebc02c6094919b3054d7da84aadf0. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:14:16.571792 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Getting quotas for user f0351b9ab5ac4fbaad38df073518412a and project d71ebc02c6094919b3054d7da84aadf0. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:14:16.582739 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Going to run 1 instances... {{(pid=117812) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:14:16.584941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='9854fd19-f34c-4ada-b3cb-25f1b62d0766',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117812) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:14:16.635105 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-bd099dc7-6f71-47a4-a293-5f14d0f35c0d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.747471 Jul 05 10:14:16.640555 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 240/512] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:14:15 2022] POST /compute/v2.1/servers => generated 380 bytes in 749 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 05 10:14:16.645948 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5cde680f-0e52-430f-ae5b-b98408eda0e6 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:16.646559 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5cde680f-0e52-430f-ae5b-b98408eda0e6 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:16.729197 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5cde680f-0e52-430f-ae5b-b98408eda0e6 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:16.744236 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5cde680f-0e52-430f-ae5b-b98408eda0e6 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1257 microversion: 2.1 time: 0.101810 Jul 05 10:14:16.764694 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 273/513] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:16 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1257 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:17.769705 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-02699206-d960-4ca0-8cfd-21b13143033f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:17.770421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-02699206-d960-4ca0-8cfd-21b13143033f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:17.778286 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-02699206-d960-4ca0-8cfd-21b13143033f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:17.779075 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-02699206-d960-4ca0-8cfd-21b13143033f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:17.780128 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-02699206-d960-4ca0-8cfd-21b13143033f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:17.780756 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-02699206-d960-4ca0-8cfd-21b13143033f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:17.785324 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=99 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:17.852457 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-02699206-d960-4ca0-8cfd-21b13143033f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:17.861163 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-02699206-d960-4ca0-8cfd-21b13143033f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1309 microversion: 2.1 time: 0.095742 Jul 05 10:14:17.861743 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 241/514] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:17 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1309 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:18.124075 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-78471739-af89-44f8-a9ae-2648e0b6c1e9 req-1e541333-c031-41ab-8c95-e4e405091ce8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4684ec3d-0407-461f-8a19-d6d8f99ca49b", "tag": "1a0fe351-4265-449d-8b7d-508a71c44b30"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:14:18.151593 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-78471739-af89-44f8-a9ae-2648e0b6c1e9 req-1e541333-c031-41ab-8c95-e4e405091ce8 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:18.152279 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-78471739-af89-44f8-a9ae-2648e0b6c1e9 req-1e541333-c031-41ab-8c95-e4e405091ce8 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:18.170663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-78471739-af89-44f8-a9ae-2648e0b6c1e9 req-1e541333-c031-41ab-8c95-e4e405091ce8 service nova] Creating event network-changed:1a0fe351-4265-449d-8b7d-508a71c44b30 for instance 4684ec3d-0407-461f-8a19-d6d8f99ca49b on hv-cinder-maste Jul 05 10:14:18.176684 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-78471739-af89-44f8-a9ae-2648e0b6c1e9 req-1e541333-c031-41ab-8c95-e4e405091ce8 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058298 Jul 05 10:14:18.177753 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 274/515] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:14:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:14:18.878048 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fdf35509-1c70-4412-9076-a58f968e7a97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:18.879272 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fdf35509-1c70-4412-9076-a58f968e7a97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:18.894034 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fdf35509-1c70-4412-9076-a58f968e7a97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:18.894747 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fdf35509-1c70-4412-9076-a58f968e7a97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:18.895870 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fdf35509-1c70-4412-9076-a58f968e7a97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:18.896545 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fdf35509-1c70-4412-9076-a58f968e7a97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:19.352000 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fdf35509-1c70-4412-9076-a58f968e7a97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:19.362013 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fdf35509-1c70-4412-9076-a58f968e7a97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.487507 Jul 05 10:14:19.365646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 242/516] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:18 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:20.380008 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ae096766-e382-4d53-99b8-397e584129a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:20.380816 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ae096766-e382-4d53-99b8-397e584129a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:20.387308 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae096766-e382-4d53-99b8-397e584129a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:20.387837 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae096766-e382-4d53-99b8-397e584129a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:20.388706 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae096766-e382-4d53-99b8-397e584129a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:20.389198 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae096766-e382-4d53-99b8-397e584129a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:20.538421 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ae096766-e382-4d53-99b8-397e584129a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:20.549794 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ae096766-e382-4d53-99b8-397e584129a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.173862 Jul 05 10:14:20.550469 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 275/517] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:20 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:21.567029 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-73fe4a49-e61a-44bf-98a8-1c88231c0e8f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:21.567752 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-73fe4a49-e61a-44bf-98a8-1c88231c0e8f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:21.574318 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-73fe4a49-e61a-44bf-98a8-1c88231c0e8f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:21.574771 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-73fe4a49-e61a-44bf-98a8-1c88231c0e8f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:21.575539 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-73fe4a49-e61a-44bf-98a8-1c88231c0e8f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:21.575980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-73fe4a49-e61a-44bf-98a8-1c88231c0e8f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:21.714207 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-73fe4a49-e61a-44bf-98a8-1c88231c0e8f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:21.721996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-73fe4a49-e61a-44bf-98a8-1c88231c0e8f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.158743 Jul 05 10:14:21.722570 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 243/518] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:21 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:22.738962 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0acc1e94-3d51-4245-98ee-b1e9158909cd tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:22.739670 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0acc1e94-3d51-4245-98ee-b1e9158909cd tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:22.748820 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0acc1e94-3d51-4245-98ee-b1e9158909cd tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:22.754758 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0acc1e94-3d51-4245-98ee-b1e9158909cd tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:22.754758 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0acc1e94-3d51-4245-98ee-b1e9158909cd tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:22.754758 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0acc1e94-3d51-4245-98ee-b1e9158909cd tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:22.889615 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0acc1e94-3d51-4245-98ee-b1e9158909cd tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:22.897613 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0acc1e94-3d51-4245-98ee-b1e9158909cd tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.162347 Jul 05 10:14:22.898209 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 276/519] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:22 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:23.915165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f0b2ed81-48f6-47e1-b8e0-2aff1656eae5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:23.915677 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f0b2ed81-48f6-47e1-b8e0-2aff1656eae5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:23.922229 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f0b2ed81-48f6-47e1-b8e0-2aff1656eae5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:23.922689 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f0b2ed81-48f6-47e1-b8e0-2aff1656eae5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:23.923462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f0b2ed81-48f6-47e1-b8e0-2aff1656eae5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:23.923907 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f0b2ed81-48f6-47e1-b8e0-2aff1656eae5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:24.020565 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1847c932-efcd-4f68-ac43-53e8d1eda514 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:24.021099 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1847c932-efcd-4f68-ac43-53e8d1eda514 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:24.028446 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1847c932-efcd-4f68-ac43-53e8d1eda514 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:24.029240 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1847c932-efcd-4f68-ac43-53e8d1eda514 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:24.030290 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1847c932-efcd-4f68-ac43-53e8d1eda514 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:24.031004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1847c932-efcd-4f68-ac43-53e8d1eda514 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:24.055626 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1847c932-efcd-4f68-ac43-53e8d1eda514 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b/ips" status: 200 len: 150 microversion: 2.1 time: 0.039023 Jul 05 10:14:24.056454 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 277/520] 10.222.0.20 () {58 vars in 1247 bytes} [Tue Jul 5 10:14:24 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b/ips => generated 150 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:14:24.064678 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f0b2ed81-48f6-47e1-b8e0-2aff1656eae5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:24.073429 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f0b2ed81-48f6-47e1-b8e0-2aff1656eae5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.162248 Jul 05 10:14:24.074008 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 244/521] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:23 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:25.090982 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1a38273a-0458-4846-959f-158778e5b24b tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:25.091595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1a38273a-0458-4846-959f-158778e5b24b tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:25.097801 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1a38273a-0458-4846-959f-158778e5b24b tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:25.098247 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1a38273a-0458-4846-959f-158778e5b24b tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:25.099000 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1a38273a-0458-4846-959f-158778e5b24b tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:25.099487 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1a38273a-0458-4846-959f-158778e5b24b tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:25.240898 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1a38273a-0458-4846-959f-158778e5b24b tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:25.249435 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1a38273a-0458-4846-959f-158778e5b24b tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.163108 Jul 05 10:14:25.250013 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 278/522] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:25 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:26.266902 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c8d346e5-a7b2-4d6c-b622-1747b5739d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:26.267537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-c8d346e5-a7b2-4d6c-b622-1747b5739d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:26.274014 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c8d346e5-a7b2-4d6c-b622-1747b5739d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:26.274467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c8d346e5-a7b2-4d6c-b622-1747b5739d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:26.275230 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c8d346e5-a7b2-4d6c-b622-1747b5739d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:26.275669 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c8d346e5-a7b2-4d6c-b622-1747b5739d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:26.405566 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-c8d346e5-a7b2-4d6c-b622-1747b5739d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:26.417645 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-c8d346e5-a7b2-4d6c-b622-1747b5739d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.152627 Jul 05 10:14:26.422719 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 245/523] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:26 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:26.492417 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8009cb2a-2bc7-4fb4-b92b-76e6ab7cf6a1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "fc1afd39-9160-4b8f-b920-a5c714959b0e"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:14:26.493474 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8009cb2a-2bc7-4fb4-b92b-76e6ab7cf6a1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:26.499965 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8009cb2a-2bc7-4fb4-b92b-76e6ab7cf6a1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:26.500424 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8009cb2a-2bc7-4fb4-b92b-76e6ab7cf6a1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:26.501192 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8009cb2a-2bc7-4fb4-b92b-76e6ab7cf6a1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:26.501618 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8009cb2a-2bc7-4fb4-b92b-76e6ab7cf6a1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:26.733900 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:27.433391 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-22e2f609-aa3f-4494-a71a-0ea71d82a206 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:27.434101 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-22e2f609-aa3f-4494-a71a-0ea71d82a206 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:27.436375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:27.437145 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=10 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:27.444680 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22e2f609-aa3f-4494-a71a-0ea71d82a206 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:27.445258 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22e2f609-aa3f-4494-a71a-0ea71d82a206 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:27.446263 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22e2f609-aa3f-4494-a71a-0ea71d82a206 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:27.446784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22e2f609-aa3f-4494-a71a-0ea71d82a206 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:27.583759 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-22e2f609-aa3f-4494-a71a-0ea71d82a206 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:27.593485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-22e2f609-aa3f-4494-a71a-0ea71d82a206 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.164804 Jul 05 10:14:27.594460 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 246/524] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:27 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:14:28.612601 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-49878f3b-53e9-4c0f-b4fa-ef369d2c138d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:28.614351 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-49878f3b-53e9-4c0f-b4fa-ef369d2c138d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:28.623319 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-49878f3b-53e9-4c0f-b4fa-ef369d2c138d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:28.624127 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-49878f3b-53e9-4c0f-b4fa-ef369d2c138d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:28.625169 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-49878f3b-53e9-4c0f-b4fa-ef369d2c138d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:28.625823 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-49878f3b-53e9-4c0f-b4fa-ef369d2c138d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:28.765870 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-49878f3b-53e9-4c0f-b4fa-ef369d2c138d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:28.774240 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-49878f3b-53e9-4c0f-b4fa-ef369d2c138d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.166597 Jul 05 10:14:28.774810 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 247/525] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:28 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:14:28.779962 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8be3bbf4-caca-4d6a-b30e-8f38a68cf520 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "78bb565d-e315-4be0-91f3-a7c8e3455b42", "device": "/dev/sdb"}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:14:28.781665 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-8be3bbf4-caca-4d6a-b30e-8f38a68cf520 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:28.787529 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8be3bbf4-caca-4d6a-b30e-8f38a68cf520 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:28.787991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8be3bbf4-caca-4d6a-b30e-8f38a68cf520 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:28.788802 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8be3bbf4-caca-4d6a-b30e-8f38a68cf520 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:28.789248 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8be3bbf4-caca-4d6a-b30e-8f38a68cf520 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:36.497362 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=27 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:36.576709 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=74 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:38.787537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=4 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:38.876647 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=47 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:14:41.474487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-8be3bbf4-caca-4d6a-b30e-8f38a68cf520 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "POST /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 12.698257 Jul 05 10:14:41.479733 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 248/526] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:14:28 2022] POST /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d/os-volume_attachments => generated 194 bytes in 12700 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:14:41.483206 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fb51da0a-d750-4968-acd5-9a31f8cef239 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:41.483714 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fb51da0a-d750-4968-acd5-9a31f8cef239 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:41.492868 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb51da0a-d750-4968-acd5-9a31f8cef239 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:41.493188 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb51da0a-d750-4968-acd5-9a31f8cef239 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:41.494096 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb51da0a-d750-4968-acd5-9a31f8cef239 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:41.494545 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb51da0a-d750-4968-acd5-9a31f8cef239 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:41.530773 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8009cb2a-2bc7-4fb4-b92b-76e6ab7cf6a1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "POST /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 15.041563 Jul 05 10:14:41.531952 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 279/527] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:14:26 2022] POST /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b/os-volume_attachments => generated 194 bytes in 15044 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:14:41.811330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fb51da0a-d750-4968-acd5-9a31f8cef239 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:41.819251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fb51da0a-d750-4968-acd5-9a31f8cef239 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.342470 Jul 05 10:14:41.819827 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 249/528] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:41 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:42.255839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-738ef031-01f3-4c9c-a8eb-f986078c898f req-13b2d659-0960-4ca1-ad25-b8b6e56024a9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5377ac3e-4f33-4b05-bc85-80b0d7b653c4", "tag": "080193f7-c96f-450e-87cc-cab454029efe"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:14:42.269284 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-738ef031-01f3-4c9c-a8eb-f986078c898f req-13b2d659-0960-4ca1-ad25-b8b6e56024a9 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:42.269731 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-738ef031-01f3-4c9c-a8eb-f986078c898f req-13b2d659-0960-4ca1-ad25-b8b6e56024a9 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:42.287319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-738ef031-01f3-4c9c-a8eb-f986078c898f req-13b2d659-0960-4ca1-ad25-b8b6e56024a9 service nova] Creating event network-changed:080193f7-c96f-450e-87cc-cab454029efe for instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 on hv-cinder-maste Jul 05 10:14:42.292747 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-738ef031-01f3-4c9c-a8eb-f986078c898f req-13b2d659-0960-4ca1-ad25-b8b6e56024a9 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048828 Jul 05 10:14:42.293352 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 280/529] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:14:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:14:42.835053 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e572db24-c403-43d1-a7d8-383d4d67c915 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:42.835569 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e572db24-c403-43d1-a7d8-383d4d67c915 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:42.842939 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e572db24-c403-43d1-a7d8-383d4d67c915 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:42.843389 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e572db24-c403-43d1-a7d8-383d4d67c915 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:42.844187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e572db24-c403-43d1-a7d8-383d4d67c915 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:42.844658 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e572db24-c403-43d1-a7d8-383d4d67c915 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:43.050585 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e572db24-c403-43d1-a7d8-383d4d67c915 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:43.058492 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e572db24-c403-43d1-a7d8-383d4d67c915 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.226813 Jul 05 10:14:43.059058 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 250/530] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:42 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:44.075388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2c210501-1af6-4b1b-b6dd-96780fdc8211 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:44.076134 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2c210501-1af6-4b1b-b6dd-96780fdc8211 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:44.082864 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2c210501-1af6-4b1b-b6dd-96780fdc8211 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:44.083313 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2c210501-1af6-4b1b-b6dd-96780fdc8211 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:44.084068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2c210501-1af6-4b1b-b6dd-96780fdc8211 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:44.084593 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2c210501-1af6-4b1b-b6dd-96780fdc8211 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:44.218128 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2c210501-1af6-4b1b-b6dd-96780fdc8211 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:44.225440 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2c210501-1af6-4b1b-b6dd-96780fdc8211 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.154215 Jul 05 10:14:44.225997 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 281/531] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:44 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:45.242987 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b8531de8-6ed4-46fb-89b6-4f0c1953d642 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:45.243649 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b8531de8-6ed4-46fb-89b6-4f0c1953d642 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:45.250629 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b8531de8-6ed4-46fb-89b6-4f0c1953d642 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:45.251084 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b8531de8-6ed4-46fb-89b6-4f0c1953d642 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:45.251849 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b8531de8-6ed4-46fb-89b6-4f0c1953d642 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:45.252296 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b8531de8-6ed4-46fb-89b6-4f0c1953d642 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:45.377334 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b8531de8-6ed4-46fb-89b6-4f0c1953d642 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:45.384605 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b8531de8-6ed4-46fb-89b6-4f0c1953d642 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.145422 Jul 05 10:14:45.385168 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 251/532] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:45 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:46.401612 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-81771137-ebaf-4a0c-baa1-f9d55497ab94 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:46.402238 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-81771137-ebaf-4a0c-baa1-f9d55497ab94 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:46.408732 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-81771137-ebaf-4a0c-baa1-f9d55497ab94 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:46.409185 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-81771137-ebaf-4a0c-baa1-f9d55497ab94 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:46.409956 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-81771137-ebaf-4a0c-baa1-f9d55497ab94 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:46.410400 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-81771137-ebaf-4a0c-baa1-f9d55497ab94 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:46.617395 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-81771137-ebaf-4a0c-baa1-f9d55497ab94 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:46.625402 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-81771137-ebaf-4a0c-baa1-f9d55497ab94 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.227636 Jul 05 10:14:46.626022 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 282/533] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:46 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:47.641607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-4023b9a5-d0fc-454e-b0fd-3764049bd459 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:47.642221 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-4023b9a5-d0fc-454e-b0fd-3764049bd459 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:47.648772 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4023b9a5-d0fc-454e-b0fd-3764049bd459 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:47.649225 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4023b9a5-d0fc-454e-b0fd-3764049bd459 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:47.649992 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4023b9a5-d0fc-454e-b0fd-3764049bd459 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:47.650428 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4023b9a5-d0fc-454e-b0fd-3764049bd459 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:47.780537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-4023b9a5-d0fc-454e-b0fd-3764049bd459 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:47.787900 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-4023b9a5-d0fc-454e-b0fd-3764049bd459 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.150061 Jul 05 10:14:47.788639 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 252/534] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:47 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:48.124268 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-352d30a6-c0d3-4963-8bef-fd79d2d96632 req-cdaa43da-045b-48e9-8aba-7849667c535d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5377ac3e-4f33-4b05-bc85-80b0d7b653c4", "name": "network-vif-plugged", "status": "completed", "tag": "080193f7-c96f-450e-87cc-cab454029efe"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:14:48.133103 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-352d30a6-c0d3-4963-8bef-fd79d2d96632 req-cdaa43da-045b-48e9-8aba-7849667c535d service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:48.133543 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-352d30a6-c0d3-4963-8bef-fd79d2d96632 req-cdaa43da-045b-48e9-8aba-7849667c535d service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:48.145041 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-352d30a6-c0d3-4963-8bef-fd79d2d96632 req-cdaa43da-045b-48e9-8aba-7849667c535d service nova] Creating event network-vif-plugged:080193f7-c96f-450e-87cc-cab454029efe for instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 on hv-cinder-maste Jul 05 10:14:48.149742 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-352d30a6-c0d3-4963-8bef-fd79d2d96632 req-cdaa43da-045b-48e9-8aba-7849667c535d service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029209 Jul 05 10:14:48.150328 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 283/535] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:14:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:14:48.804833 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-009c2e54-0817-44a9-8b8f-3bf1fc03cae0 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:48.805467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-009c2e54-0817-44a9-8b8f-3bf1fc03cae0 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:48.812378 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-009c2e54-0817-44a9-8b8f-3bf1fc03cae0 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:48.812857 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-009c2e54-0817-44a9-8b8f-3bf1fc03cae0 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:48.813629 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-009c2e54-0817-44a9-8b8f-3bf1fc03cae0 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:48.814233 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-009c2e54-0817-44a9-8b8f-3bf1fc03cae0 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:49.002385 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-009c2e54-0817-44a9-8b8f-3bf1fc03cae0 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:49.010367 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-009c2e54-0817-44a9-8b8f-3bf1fc03cae0 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.209370 Jul 05 10:14:49.011450 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 253/536] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:48 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:50.028367 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-cbbec836-82d2-4ecf-b85b-e4259cc9575a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:50.029160 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-cbbec836-82d2-4ecf-b85b-e4259cc9575a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:50.035765 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cbbec836-82d2-4ecf-b85b-e4259cc9575a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:50.036292 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cbbec836-82d2-4ecf-b85b-e4259cc9575a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:50.037159 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cbbec836-82d2-4ecf-b85b-e4259cc9575a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:50.037674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cbbec836-82d2-4ecf-b85b-e4259cc9575a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:50.234036 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-cbbec836-82d2-4ecf-b85b-e4259cc9575a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:50.243529 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-cbbec836-82d2-4ecf-b85b-e4259cc9575a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.220347 Jul 05 10:14:50.244245 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 284/537] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:50 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:51.260838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a09cfa9d-329f-4a5e-9ebf-92efe19df2d4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:51.261560 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a09cfa9d-329f-4a5e-9ebf-92efe19df2d4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:51.267860 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a09cfa9d-329f-4a5e-9ebf-92efe19df2d4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:51.268308 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a09cfa9d-329f-4a5e-9ebf-92efe19df2d4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:51.269109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a09cfa9d-329f-4a5e-9ebf-92efe19df2d4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:51.269549 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a09cfa9d-329f-4a5e-9ebf-92efe19df2d4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:51.427136 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a09cfa9d-329f-4a5e-9ebf-92efe19df2d4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:51.447922 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a09cfa9d-329f-4a5e-9ebf-92efe19df2d4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.190929 Jul 05 10:14:51.448537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 254/538] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:51 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:52.463900 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a6fa394a-d5f7-4109-b7f0-7ac86135b303 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:52.464527 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a6fa394a-d5f7-4109-b7f0-7ac86135b303 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:52.471179 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a6fa394a-d5f7-4109-b7f0-7ac86135b303 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:52.471622 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a6fa394a-d5f7-4109-b7f0-7ac86135b303 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:52.472379 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a6fa394a-d5f7-4109-b7f0-7ac86135b303 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:52.472833 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a6fa394a-d5f7-4109-b7f0-7ac86135b303 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:52.598808 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a6fa394a-d5f7-4109-b7f0-7ac86135b303 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:52.606571 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a6fa394a-d5f7-4109-b7f0-7ac86135b303 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.146406 Jul 05 10:14:52.607125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 285/539] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:52 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:53.623416 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f9dbbad3-5bce-4f3d-9661-b12c0ef8c00d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:53.624027 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f9dbbad3-5bce-4f3d-9661-b12c0ef8c00d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:53.630869 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f9dbbad3-5bce-4f3d-9661-b12c0ef8c00d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:53.631328 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f9dbbad3-5bce-4f3d-9661-b12c0ef8c00d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:53.632090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f9dbbad3-5bce-4f3d-9661-b12c0ef8c00d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:53.632555 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f9dbbad3-5bce-4f3d-9661-b12c0ef8c00d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:53.757823 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f9dbbad3-5bce-4f3d-9661-b12c0ef8c00d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:53.766416 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f9dbbad3-5bce-4f3d-9661-b12c0ef8c00d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.147353 Jul 05 10:14:53.766982 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 255/540] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:53 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:54.782885 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0d4acfd6-87f2-4176-8fba-bdd2df5480d6 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:54.783493 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0d4acfd6-87f2-4176-8fba-bdd2df5480d6 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:54.789988 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0d4acfd6-87f2-4176-8fba-bdd2df5480d6 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:54.790433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0d4acfd6-87f2-4176-8fba-bdd2df5480d6 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:54.791192 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0d4acfd6-87f2-4176-8fba-bdd2df5480d6 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:54.791628 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0d4acfd6-87f2-4176-8fba-bdd2df5480d6 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:54.925494 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0d4acfd6-87f2-4176-8fba-bdd2df5480d6 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:54.939582 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0d4acfd6-87f2-4176-8fba-bdd2df5480d6 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1357 microversion: 2.1 time: 0.160464 Jul 05 10:14:54.940164 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 286/541] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:54 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1357 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:14:55.957648 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f46764eb-6d7a-47b2-9e6f-233afbb2001c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:55.958257 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f46764eb-6d7a-47b2-9e6f-233afbb2001c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:55.964724 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f46764eb-6d7a-47b2-9e6f-233afbb2001c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:55.965167 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f46764eb-6d7a-47b2-9e6f-233afbb2001c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:55.965926 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f46764eb-6d7a-47b2-9e6f-233afbb2001c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:55.966615 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f46764eb-6d7a-47b2-9e6f-233afbb2001c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:56.090591 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f46764eb-6d7a-47b2-9e6f-233afbb2001c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:56.098098 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f46764eb-6d7a-47b2-9e6f-233afbb2001c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1547 microversion: 2.1 time: 0.144580 Jul 05 10:14:56.098660 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 256/542] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:55 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1547 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:56.113906 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e9871f6d-0265-4b31-8fdd-29d4c135e257 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:56.114508 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e9871f6d-0265-4b31-8fdd-29d4c135e257 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:56.121522 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e9871f6d-0265-4b31-8fdd-29d4c135e257 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:56.121968 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e9871f6d-0265-4b31-8fdd-29d4c135e257 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:56.122742 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e9871f6d-0265-4b31-8fdd-29d4c135e257 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:56.123174 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e9871f6d-0265-4b31-8fdd-29d4c135e257 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:56.251784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e9871f6d-0265-4b31-8fdd-29d4c135e257 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:56.259365 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e9871f6d-0265-4b31-8fdd-29d4c135e257 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1547 microversion: 2.1 time: 0.149199 Jul 05 10:14:56.259930 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 287/543] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:56 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1547 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:14:56.275239 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e165efe0-9a44-4d5f-ae34-d41b5fcd4cd3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:56.279341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e165efe0-9a44-4d5f-ae34-d41b5fcd4cd3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:56.280183 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e165efe0-9a44-4d5f-ae34-d41b5fcd4cd3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Searching by: {'deleted': False, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0'} {{(pid=117812) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 05 10:14:56.287007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e165efe0-9a44-4d5f-ae34-d41b5fcd4cd3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:56.287462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e165efe0-9a44-4d5f-ae34-d41b5fcd4cd3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:56.288245 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e165efe0-9a44-4d5f-ae34-d41b5fcd4cd3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:56.288710 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e165efe0-9a44-4d5f-ae34-d41b5fcd4cd3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:56.379150 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.multi_cell_list [None req-e165efe0-9a44-4d5f-ae34-d41b5fcd4cd3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=117812) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 05 10:14:56.475778 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e165efe0-9a44-4d5f-ae34-d41b5fcd4cd3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.204163 Jul 05 10:14:56.476271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 257/544] 10.222.0.20 () {58 vars in 1124 bytes} [Tue Jul 5 10:14:56 2022] GET /compute/v2.1/servers => generated 342 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:14:56.486498 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-392f74f9-7c82-4a81-8bb6-f8b7e6bba31c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:14:56.487105 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-392f74f9-7c82-4a81-8bb6-f8b7e6bba31c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:14:56.493060 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-392f74f9-7c82-4a81-8bb6-f8b7e6bba31c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:56.493509 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-392f74f9-7c82-4a81-8bb6-f8b7e6bba31c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:56.494270 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-392f74f9-7c82-4a81-8bb6-f8b7e6bba31c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:14:56.494701 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-392f74f9-7c82-4a81-8bb6-f8b7e6bba31c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:14:56.622694 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-392f74f9-7c82-4a81-8bb6-f8b7e6bba31c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:14:56.630204 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-392f74f9-7c82-4a81-8bb6-f8b7e6bba31c tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1547 microversion: 2.1 time: 0.147577 Jul 05 10:14:56.630875 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 288/545] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:14:56 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1547 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:01.701305 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0e3275dd-01a6-4649-9a63-a737ecea30b5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Action: 'create', calling method: .version_select at 0x7feb89864310>, body: {"keypair": {"name": "tempest-keypair-2076576628", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAog94zKe5ADJ7qWkxk8KP/AnPD4glkMChxF3wWqIQbRbO2Pz4K4I+k9xlp1pyfD3jNAEqo5KZ6HXEc++DCTl8qOlplXDSZTEo0PN1am/CoMd0wA2HWuAmIbZAKeiYjyZQ=="}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:01.716979 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-0e3275dd-01a6-4649-9a63-a737ecea30b5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:15:01.720886 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-0e3275dd-01a6-4649-9a63-a737ecea30b5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for user 036cb887c7894e75ba3eb4c1a8b21c69 and project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:15:01.741670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0e3275dd-01a6-4649-9a63-a737ecea30b5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.044740 Jul 05 10:15:01.742250 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 258/546] 10.222.0.20 () {60 vars in 1158 bytes} [Tue Jul 5 10:15:01 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:15:02.316360 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-80d25f82-7d15-4192-b025-f82f188077ad tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "69d67d51-ed91-43a9-9f0e-2c5c195d34c6"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:02.317445 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-80d25f82-7d15-4192-b025-f82f188077ad tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:02.323871 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80d25f82-7d15-4192-b025-f82f188077ad tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:02.324346 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80d25f82-7d15-4192-b025-f82f188077ad tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:02.325128 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80d25f82-7d15-4192-b025-f82f188077ad tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:02.325565 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80d25f82-7d15-4192-b025-f82f188077ad tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:03.248757 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-839373214", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "033e3c55-aad3-40e5-8b8b-1dedb7b873c7"}], "security_groups": [{"name": "tempest-securitygroup--547668594"}], "key_name": "tempest-keypair-2076576628", "user_data": "}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:03.349696 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.network.neutron [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] validate_networks() for [('033e3c55-aad3-40e5-8b8b-1dedb7b873c7', None, None, None, None, None)] {{(pid=117812) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:15:03.707682 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Flavor limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:15:03.708209 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Image limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:15:03.708712 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Flavor pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:15:03.709118 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Image pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:15:03.709522 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:15:03.718755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:03.719214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:03.720050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:03.720529 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:03.736828 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:15:03.740755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for user 036cb887c7894e75ba3eb4c1a8b21c69 and project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:15:03.753443 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Going to run 1 instances... {{(pid=117812) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:15:03.755736 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117812) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:15:03.801267 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0c748e76-0e00-422e-8685-a44c7555f302 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.557418 Jul 05 10:15:03.802085 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 259/547] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:15:03 2022] POST /compute/v2.1/servers => generated 401 bytes in 559 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 05 10:15:03.811885 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-19c41596-4848-4d2c-b9e4-b1bfd8742550 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:03.812518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-19c41596-4848-4d2c-b9e4-b1bfd8742550 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:03.865487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-19c41596-4848-4d2c-b9e4-b1bfd8742550 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:03.872768 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-19c41596-4848-4d2c-b9e4-b1bfd8742550 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1241 microversion: 2.1 time: 0.064960 Jul 05 10:15:03.873335 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 260/548] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:03 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1241 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:15:04.889428 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-902aca6f-58c1-45ea-bb3f-c61d1f9a6cfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:04.890075 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-902aca6f-58c1-45ea-bb3f-c61d1f9a6cfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:04.896647 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-902aca6f-58c1-45ea-bb3f-c61d1f9a6cfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:04.897103 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-902aca6f-58c1-45ea-bb3f-c61d1f9a6cfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:04.897872 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-902aca6f-58c1-45ea-bb3f-c61d1f9a6cfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:04.898315 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-902aca6f-58c1-45ea-bb3f-c61d1f9a6cfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:04.958375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-902aca6f-58c1-45ea-bb3f-c61d1f9a6cfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:04.966448 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-902aca6f-58c1-45ea-bb3f-c61d1f9a6cfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1245 microversion: 2.1 time: 0.080808 Jul 05 10:15:04.967274 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 261/549] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:04 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1245 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:15:05.983250 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ef1fc8c9-5e9c-4c82-a672-f043015e80a4 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:05.983860 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ef1fc8c9-5e9c-4c82-a672-f043015e80a4 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:05.990308 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ef1fc8c9-5e9c-4c82-a672-f043015e80a4 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:05.990761 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ef1fc8c9-5e9c-4c82-a672-f043015e80a4 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:05.991520 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ef1fc8c9-5e9c-4c82-a672-f043015e80a4 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:05.991971 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ef1fc8c9-5e9c-4c82-a672-f043015e80a4 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:06.259601 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ef1fc8c9-5e9c-4c82-a672-f043015e80a4 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:06.267250 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ef1fc8c9-5e9c-4c82-a672-f043015e80a4 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1245 microversion: 2.1 time: 0.287803 Jul 05 10:15:06.267885 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 262/550] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:05 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1245 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:15:06.330941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-80d25f82-7d15-4192-b025-f82f188077ad tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "POST /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 4.018271 Jul 05 10:15:06.331717 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 289/551] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:15:02 2022] POST /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/os-volume_attachments => generated 194 bytes in 4020 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:15:07.283688 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6034aeb1-f491-4cee-84d3-1f79950a5529 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:07.284353 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6034aeb1-f491-4cee-84d3-1f79950a5529 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:07.303103 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6034aeb1-f491-4cee-84d3-1f79950a5529 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:07.303679 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6034aeb1-f491-4cee-84d3-1f79950a5529 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:07.304748 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6034aeb1-f491-4cee-84d3-1f79950a5529 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:07.305363 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6034aeb1-f491-4cee-84d3-1f79950a5529 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:07.370791 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6034aeb1-f491-4cee-84d3-1f79950a5529 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:07.378895 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6034aeb1-f491-4cee-84d3-1f79950a5529 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1311 microversion: 2.1 time: 0.098925 Jul 05 10:15:07.379401 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 263/552] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:07 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1311 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:08.397678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-392f3a36-f6ef-4f15-b414-390408824a20 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:08.399021 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-392f3a36-f6ef-4f15-b414-390408824a20 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:08.402189 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=21 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:08.407642 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-392f3a36-f6ef-4f15-b414-390408824a20 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:08.408246 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-392f3a36-f6ef-4f15-b414-390408824a20 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:08.409163 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-392f3a36-f6ef-4f15-b414-390408824a20 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:08.409654 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-392f3a36-f6ef-4f15-b414-390408824a20 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:08.428330 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=63,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:08.659716 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-392f3a36-f6ef-4f15-b414-390408824a20 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:08.667272 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-392f3a36-f6ef-4f15-b414-390408824a20 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1366 microversion: 2.1 time: 0.274628 Jul 05 10:15:08.667936 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 290/553] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:08 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1366 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:15:09.101662 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-37e0432e-fd19-4b54-8ca0-4387d98f7660 req-f41f4b2d-6422-4740-a06e-e262dd24c4da service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a325da12-82e8-4964-a4db-4d38efa52dc7", "tag": "de7ed2e8-b690-41d8-936e-1c400d4b704d"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:09.110195 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=55,nova_api:INSERT=4 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:09.112646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-37e0432e-fd19-4b54-8ca0-4387d98f7660 req-f41f4b2d-6422-4740-a06e-e262dd24c4da service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:09.113479 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-37e0432e-fd19-4b54-8ca0-4387d98f7660 req-f41f4b2d-6422-4740-a06e-e262dd24c4da service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:09.124432 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=82,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:09.126678 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-37e0432e-fd19-4b54-8ca0-4387d98f7660 req-f41f4b2d-6422-4740-a06e-e262dd24c4da service nova] Creating event network-changed:de7ed2e8-b690-41d8-936e-1c400d4b704d for instance a325da12-82e8-4964-a4db-4d38efa52dc7 on hv-cinder-maste Jul 05 10:15:09.132134 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-37e0432e-fd19-4b54-8ca0-4387d98f7660 req-f41f4b2d-6422-4740-a06e-e262dd24c4da service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040465 Jul 05 10:15:09.133084 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 264/554] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:15:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:15:09.683530 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5b460e3a-2c0d-4ca5-b2eb-b1f778221f63 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:09.684134 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5b460e3a-2c0d-4ca5-b2eb-b1f778221f63 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:09.690550 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5b460e3a-2c0d-4ca5-b2eb-b1f778221f63 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:09.690996 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5b460e3a-2c0d-4ca5-b2eb-b1f778221f63 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:09.691801 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5b460e3a-2c0d-4ca5-b2eb-b1f778221f63 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:09.692238 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5b460e3a-2c0d-4ca5-b2eb-b1f778221f63 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:09.822350 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5b460e3a-2c0d-4ca5-b2eb-b1f778221f63 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:09.830169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5b460e3a-2c0d-4ca5-b2eb-b1f778221f63 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1366 microversion: 2.1 time: 0.150538 Jul 05 10:15:09.830726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 291/555] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:09 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1366 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:10.849549 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f2d3fc9e-00a5-41f1-b75c-aa76e9813e6e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:10.850164 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f2d3fc9e-00a5-41f1-b75c-aa76e9813e6e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:10.857687 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f2d3fc9e-00a5-41f1-b75c-aa76e9813e6e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:10.858141 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f2d3fc9e-00a5-41f1-b75c-aa76e9813e6e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:10.858908 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f2d3fc9e-00a5-41f1-b75c-aa76e9813e6e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:10.859347 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f2d3fc9e-00a5-41f1-b75c-aa76e9813e6e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:10.988631 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f2d3fc9e-00a5-41f1-b75c-aa76e9813e6e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:10.995859 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f2d3fc9e-00a5-41f1-b75c-aa76e9813e6e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1366 microversion: 2.1 time: 0.152238 Jul 05 10:15:10.996961 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 265/556] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:10 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:11.440507 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3e2f0206-0f42-498f-b6a2-a1806dcd7584 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:11.440996 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3e2f0206-0f42-498f-b6a2-a1806dcd7584 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:11.447332 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3e2f0206-0f42-498f-b6a2-a1806dcd7584 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:11.447778 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3e2f0206-0f42-498f-b6a2-a1806dcd7584 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:11.448569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3e2f0206-0f42-498f-b6a2-a1806dcd7584 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:11.449004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3e2f0206-0f42-498f-b6a2-a1806dcd7584 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:11.570069 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.objects.instance [None req-3e2f0206-0f42-498f-b6a2-a1806dcd7584 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lazy-loading 'services' on Instance uuid 86fe0729-f79b-4fee-a79a-d004a32cfa2d {{(pid=117813) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:15:11.681344 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3e2f0206-0f42-498f-b6a2-a1806dcd7584 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "DELETE /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d/os-volume_attachments/78bb565d-e315-4be0-91f3-a7c8e3455b42" status: 202 len: 0 microversion: 2.1 time: 0.244584 Jul 05 10:15:11.681937 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 292/557] 10.222.0.20 () {58 vars in 1415 bytes} [Tue Jul 5 10:15:11 2022] DELETE /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d/os-volume_attachments/78bb565d-e315-4be0-91f3-a7c8e3455b42 => generated 0 bytes in 246 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:15:12.013476 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8b3a4320-6513-4f0c-8abf-9aaac9efdc4b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:12.014045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-8b3a4320-6513-4f0c-8abf-9aaac9efdc4b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:12.020488 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8b3a4320-6513-4f0c-8abf-9aaac9efdc4b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:12.020940 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8b3a4320-6513-4f0c-8abf-9aaac9efdc4b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:12.021702 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8b3a4320-6513-4f0c-8abf-9aaac9efdc4b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:12.022138 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8b3a4320-6513-4f0c-8abf-9aaac9efdc4b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:12.153678 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8b3a4320-6513-4f0c-8abf-9aaac9efdc4b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:12.160873 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-8b3a4320-6513-4f0c-8abf-9aaac9efdc4b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1366 microversion: 2.1 time: 0.151878 Jul 05 10:15:12.161434 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 266/558] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:12 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1366 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:13.178444 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1414a5a4-bd13-4572-8a92-31e35a73a797 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:13.179254 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1414a5a4-bd13-4572-8a92-31e35a73a797 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:13.186566 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1414a5a4-bd13-4572-8a92-31e35a73a797 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:13.187016 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1414a5a4-bd13-4572-8a92-31e35a73a797 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:13.187774 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1414a5a4-bd13-4572-8a92-31e35a73a797 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:13.188209 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1414a5a4-bd13-4572-8a92-31e35a73a797 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:13.318715 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1414a5a4-bd13-4572-8a92-31e35a73a797 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:13.326793 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1414a5a4-bd13-4572-8a92-31e35a73a797 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1366 microversion: 2.1 time: 0.153211 Jul 05 10:15:13.327356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 293/559] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:13 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1366 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:14.343967 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-68114c42-11bb-42fc-910f-bc20dbbeddfe tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:14.344530 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-68114c42-11bb-42fc-910f-bc20dbbeddfe tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:14.347079 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=4 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:14.347873 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=12 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:14.353895 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-68114c42-11bb-42fc-910f-bc20dbbeddfe tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:14.354348 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-68114c42-11bb-42fc-910f-bc20dbbeddfe tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:14.355124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-68114c42-11bb-42fc-910f-bc20dbbeddfe tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:14.355562 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-68114c42-11bb-42fc-910f-bc20dbbeddfe tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:14.491836 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-68114c42-11bb-42fc-910f-bc20dbbeddfe tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:14.499441 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-68114c42-11bb-42fc-910f-bc20dbbeddfe tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1366 microversion: 2.1 time: 0.160162 Jul 05 10:15:14.500321 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 267/560] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:14 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1366 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:15.515383 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3ca708d2-b43e-410b-839a-12dc64a0e8d8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:15.515983 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3ca708d2-b43e-410b-839a-12dc64a0e8d8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:15.522387 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3ca708d2-b43e-410b-839a-12dc64a0e8d8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:15.522827 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3ca708d2-b43e-410b-839a-12dc64a0e8d8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:15.523574 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3ca708d2-b43e-410b-839a-12dc64a0e8d8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:15.524002 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3ca708d2-b43e-410b-839a-12dc64a0e8d8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:15.656320 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-3ca708d2-b43e-410b-839a-12dc64a0e8d8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:15.663742 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3ca708d2-b43e-410b-839a-12dc64a0e8d8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1366 microversion: 2.1 time: 0.152112 Jul 05 10:15:15.664313 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 294/561] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:15 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1366 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:15.981786 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e1d86e6f-c57e-4ae8-afb0-168e8fdaae9a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:15.982456 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e1d86e6f-c57e-4ae8-afb0-168e8fdaae9a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:15.989191 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e1d86e6f-c57e-4ae8-afb0-168e8fdaae9a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:15.989639 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e1d86e6f-c57e-4ae8-afb0-168e8fdaae9a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:15.990407 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e1d86e6f-c57e-4ae8-afb0-168e8fdaae9a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:15.990840 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e1d86e6f-c57e-4ae8-afb0-168e8fdaae9a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:16.149110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e1d86e6f-c57e-4ae8-afb0-168e8fdaae9a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:16.157197 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e1d86e6f-c57e-4ae8-afb0-168e8fdaae9a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1593 microversion: 2.1 time: 0.179219 Jul 05 10:15:16.157758 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 268/562] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:15 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1593 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:16.373419 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:15:16.374128 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Tue Jul 5 10:15:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:15:16.417758 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-d226a4d0-1f13-4eb1-94ab-514fe2b47d96 req-787931d5-374d-485d-b76d-f191599259cc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a325da12-82e8-4964-a4db-4d38efa52dc7", "name": "network-vif-plugged", "status": "completed", "tag": "de7ed2e8-b690-41d8-936e-1c400d4b704d"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:16.429994 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-d226a4d0-1f13-4eb1-94ab-514fe2b47d96 req-787931d5-374d-485d-b76d-f191599259cc service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:16.430957 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-d226a4d0-1f13-4eb1-94ab-514fe2b47d96 req-787931d5-374d-485d-b76d-f191599259cc service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:16.448099 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-d226a4d0-1f13-4eb1-94ab-514fe2b47d96 req-787931d5-374d-485d-b76d-f191599259cc service nova] Creating event network-vif-plugged:de7ed2e8-b690-41d8-936e-1c400d4b704d for instance a325da12-82e8-4964-a4db-4d38efa52dc7 on hv-cinder-maste Jul 05 10:15:16.457154 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-d226a4d0-1f13-4eb1-94ab-514fe2b47d96 req-787931d5-374d-485d-b76d-f191599259cc service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043458 Jul 05 10:15:16.459741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 295/563] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:15:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:15:16.681738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-36b7e584-8b28-4f03-b90f-75aeaf2aa545 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:16.682535 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-36b7e584-8b28-4f03-b90f-75aeaf2aa545 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:16.689707 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36b7e584-8b28-4f03-b90f-75aeaf2aa545 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:16.692740 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36b7e584-8b28-4f03-b90f-75aeaf2aa545 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:16.693834 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36b7e584-8b28-4f03-b90f-75aeaf2aa545 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:16.694410 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36b7e584-8b28-4f03-b90f-75aeaf2aa545 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:16.875036 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-36b7e584-8b28-4f03-b90f-75aeaf2aa545 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:16.882888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-36b7e584-8b28-4f03-b90f-75aeaf2aa545 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1366 microversion: 2.1 time: 0.205030 Jul 05 10:15:16.883465 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 269/564] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:16 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1366 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:17.014145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1c334633-dd06-4d8e-919c-edd3e0a8495a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:17.014728 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1c334633-dd06-4d8e-919c-edd3e0a8495a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:17.021238 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1c334633-dd06-4d8e-919c-edd3e0a8495a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:17.021674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1c334633-dd06-4d8e-919c-edd3e0a8495a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:17.022444 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1c334633-dd06-4d8e-919c-edd3e0a8495a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:17.022883 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1c334633-dd06-4d8e-919c-edd3e0a8495a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:17.048761 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1c334633-dd06-4d8e-919c-edd3e0a8495a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Going to try to terminate instance {{(pid=117813) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:15:17.121846 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1c334633-dd06-4d8e-919c-edd3e0a8495a tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "DELETE /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 204 len: 0 microversion: 2.1 time: 0.112096 Jul 05 10:15:17.122518 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 296/565] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:15:17 2022] DELETE /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:15:17.131081 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b5542d76-07a1-40c0-9dc3-3375cf85908e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:17.132245 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b5542d76-07a1-40c0-9dc3-3375cf85908e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:17.138681 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b5542d76-07a1-40c0-9dc3-3375cf85908e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:17.139117 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b5542d76-07a1-40c0-9dc3-3375cf85908e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:17.139882 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b5542d76-07a1-40c0-9dc3-3375cf85908e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:17.140317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b5542d76-07a1-40c0-9dc3-3375cf85908e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:17.276201 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b5542d76-07a1-40c0-9dc3-3375cf85908e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:17.283710 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b5542d76-07a1-40c0-9dc3-3375cf85908e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1513 microversion: 2.1 time: 0.157077 Jul 05 10:15:17.284271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 270/566] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:17 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1513 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:17.900224 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3a569cde-032a-40ec-89dc-3df829bb13b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:17.901358 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3a569cde-032a-40ec-89dc-3df829bb13b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:17.915897 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3a569cde-032a-40ec-89dc-3df829bb13b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:17.916366 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3a569cde-032a-40ec-89dc-3df829bb13b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:17.917835 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3a569cde-032a-40ec-89dc-3df829bb13b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:17.917835 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3a569cde-032a-40ec-89dc-3df829bb13b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:17.918214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-9f79dafd-2902-4f70-9050-c1b5a5aae495 req-6350e419-4e3e-454b-8000-6e2c14130247 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5377ac3e-4f33-4b05-bc85-80b0d7b653c4", "tag": "080193f7-c96f-450e-87cc-cab454029efe"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:17.927532 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-9f79dafd-2902-4f70-9050-c1b5a5aae495 req-6350e419-4e3e-454b-8000-6e2c14130247 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:17.928320 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-9f79dafd-2902-4f70-9050-c1b5a5aae495 req-6350e419-4e3e-454b-8000-6e2c14130247 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:17.943469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-9f79dafd-2902-4f70-9050-c1b5a5aae495 req-6350e419-4e3e-454b-8000-6e2c14130247 service nova] Creating event network-changed:080193f7-c96f-450e-87cc-cab454029efe for instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 on hv-cinder-maste Jul 05 10:15:17.949469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-9f79dafd-2902-4f70-9050-c1b5a5aae495 req-6350e419-4e3e-454b-8000-6e2c14130247 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034795 Jul 05 10:15:17.950171 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 271/567] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:15:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:15:18.116202 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-3a569cde-032a-40ec-89dc-3df829bb13b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:18.124241 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3a569cde-032a-40ec-89dc-3df829bb13b6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1366 microversion: 2.1 time: 0.228419 Jul 05 10:15:18.124947 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 297/568] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:17 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1366 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:18.302396 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c9d273a8-4f8b-461d-9867-b2019b5c146d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:18.303804 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-c9d273a8-4f8b-461d-9867-b2019b5c146d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:18.311230 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c9d273a8-4f8b-461d-9867-b2019b5c146d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:18.312829 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c9d273a8-4f8b-461d-9867-b2019b5c146d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:18.313630 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c9d273a8-4f8b-461d-9867-b2019b5c146d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:18.314064 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c9d273a8-4f8b-461d-9867-b2019b5c146d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:18.494673 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-c9d273a8-4f8b-461d-9867-b2019b5c146d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:18.504787 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-c9d273a8-4f8b-461d-9867-b2019b5c146d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1513 microversion: 2.1 time: 0.206883 Jul 05 10:15:18.514495 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 272/569] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:18 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1513 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:19.008737 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a09aee2f-4ffd-47de-ae73-ab4f3052b542 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-179336894"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:19.009337 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a09aee2f-4ffd-47de-ae73-ab4f3052b542 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:19.016303 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a09aee2f-4ffd-47de-ae73-ab4f3052b542 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:19.016888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a09aee2f-4ffd-47de-ae73-ab4f3052b542 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:19.017706 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a09aee2f-4ffd-47de-ae73-ab4f3052b542 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:19.018307 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a09aee2f-4ffd-47de-ae73-ab4f3052b542 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:19.152762 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.network.security_group_api [None req-a09aee2f-4ffd-47de-ae73-ab4f3052b542 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Adding security group ec33f5ae-fb77-4249-a272-9317a9186cf4 to port 080193f7-c96f-450e-87cc-cab454029efe Jul 05 10:15:19.194759 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ac0b9765-be62-47f4-a64f-02565f55ac64 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:19.195367 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ac0b9765-be62-47f4-a64f-02565f55ac64 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:19.201945 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ac0b9765-be62-47f4-a64f-02565f55ac64 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:19.202395 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ac0b9765-be62-47f4-a64f-02565f55ac64 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:19.203155 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ac0b9765-be62-47f4-a64f-02565f55ac64 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:19.203593 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ac0b9765-be62-47f4-a64f-02565f55ac64 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:19.406999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ac0b9765-be62-47f4-a64f-02565f55ac64 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:19.414296 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ac0b9765-be62-47f4-a64f-02565f55ac64 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1366 microversion: 2.1 time: 0.277388 Jul 05 10:15:19.414864 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 273/570] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:19 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1366 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:19.521674 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fb0d5ff3-027b-44d5-aaff-b32b0ff2f334 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:19.522328 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fb0d5ff3-027b-44d5-aaff-b32b0ff2f334 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:19.528469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb0d5ff3-027b-44d5-aaff-b32b0ff2f334 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:19.529551 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb0d5ff3-027b-44d5-aaff-b32b0ff2f334 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:19.530324 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb0d5ff3-027b-44d5-aaff-b32b0ff2f334 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:19.530764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb0d5ff3-027b-44d5-aaff-b32b0ff2f334 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:19.577782 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a09aee2f-4ffd-47de-ae73-ab4f3052b542 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "POST /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/action" status: 202 len: 0 microversion: 2.1 time: 0.572821 Jul 05 10:15:19.582353 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 298/571] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:15:19 2022] POST /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/action => generated 0 bytes in 578 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:15:19.587866 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-b59461f5-7566-40ef-b427-49266a2f7299 req-058aef79-fdcb-4b09-a407-d44a4b5ac9e9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5377ac3e-4f33-4b05-bc85-80b0d7b653c4", "tag": "080193f7-c96f-450e-87cc-cab454029efe"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:19.603348 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-b59461f5-7566-40ef-b427-49266a2f7299 req-058aef79-fdcb-4b09-a407-d44a4b5ac9e9 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:19.604077 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-b59461f5-7566-40ef-b427-49266a2f7299 req-058aef79-fdcb-4b09-a407-d44a4b5ac9e9 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:19.618189 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-b59461f5-7566-40ef-b427-49266a2f7299 req-058aef79-fdcb-4b09-a407-d44a4b5ac9e9 service nova] Creating event network-changed:080193f7-c96f-450e-87cc-cab454029efe for instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 on hv-cinder-maste Jul 05 10:15:19.625952 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-b59461f5-7566-40ef-b427-49266a2f7299 req-058aef79-fdcb-4b09-a407-d44a4b5ac9e9 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042121 Jul 05 10:15:19.628100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 299/572] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:15:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:15:19.636014 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-56b0b67e-4c00-4114-b2c0-43d8c56f1ce8 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:19.638865 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-56b0b67e-4c00-4114-b2c0-43d8c56f1ce8 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:19.647940 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-56b0b67e-4c00-4114-b2c0-43d8c56f1ce8 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:19.648653 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-56b0b67e-4c00-4114-b2c0-43d8c56f1ce8 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:19.649986 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-56b0b67e-4c00-4114-b2c0-43d8c56f1ce8 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:19.650639 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-56b0b67e-4c00-4114-b2c0-43d8c56f1ce8 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:19.735421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fb0d5ff3-027b-44d5-aaff-b32b0ff2f334 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:19.744969 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fb0d5ff3-027b-44d5-aaff-b32b0ff2f334 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1513 microversion: 2.1 time: 0.226855 Jul 05 10:15:19.745915 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 274/573] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:19 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1513 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:15:19.820070 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-56b0b67e-4c00-4114-b2c0-43d8c56f1ce8 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:19.828595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-56b0b67e-4c00-4114-b2c0-43d8c56f1ce8 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1639 microversion: 2.1 time: 0.198952 Jul 05 10:15:19.829155 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 300/574] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:19 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1639 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:20.431445 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f3f2e5b4-20bc-417d-b7bf-796a213e7c3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:20.432055 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f3f2e5b4-20bc-417d-b7bf-796a213e7c3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:20.439011 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f3f2e5b4-20bc-417d-b7bf-796a213e7c3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:20.439452 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f3f2e5b4-20bc-417d-b7bf-796a213e7c3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:20.440202 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f3f2e5b4-20bc-417d-b7bf-796a213e7c3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:20.440660 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f3f2e5b4-20bc-417d-b7bf-796a213e7c3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:20.566242 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f3f2e5b4-20bc-417d-b7bf-796a213e7c3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:20.573717 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f3f2e5b4-20bc-417d-b7bf-796a213e7c3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1366 microversion: 2.1 time: 0.146121 Jul 05 10:15:20.574281 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 275/575] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:20 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1366 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:20.763569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a22b102e-27c5-40a1-b834-6f1f5d42db5c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:20.764084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a22b102e-27c5-40a1-b834-6f1f5d42db5c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:20.770478 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a22b102e-27c5-40a1-b834-6f1f5d42db5c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:20.770919 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a22b102e-27c5-40a1-b834-6f1f5d42db5c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:20.771667 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a22b102e-27c5-40a1-b834-6f1f5d42db5c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:20.772099 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a22b102e-27c5-40a1-b834-6f1f5d42db5c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:20.908986 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a22b102e-27c5-40a1-b834-6f1f5d42db5c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:20.916967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a22b102e-27c5-40a1-b834-6f1f5d42db5c tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1513 microversion: 2.1 time: 0.158335 Jul 05 10:15:20.928156 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 301/576] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:20 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1513 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:21.590506 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-36985ee1-179f-4bee-83f0-b69bb4388cd3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:21.591121 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-36985ee1-179f-4bee-83f0-b69bb4388cd3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:21.598007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36985ee1-179f-4bee-83f0-b69bb4388cd3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:21.598456 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36985ee1-179f-4bee-83f0-b69bb4388cd3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:21.599225 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36985ee1-179f-4bee-83f0-b69bb4388cd3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:21.599659 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36985ee1-179f-4bee-83f0-b69bb4388cd3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:21.885223 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-cb8f74bc-3644-4b98-a88b-1b345ffd53a8 req-641e731a-7413-4a0c-a847-d4d30b6b4f10 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "86fe0729-f79b-4fee-a79a-d004a32cfa2d", "name": "network-vif-deleted", "tag": "b00d9c04-1d84-4211-8344-23f678f0b868"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:21.888952 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-36985ee1-179f-4bee-83f0-b69bb4388cd3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:21.896749 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-cb8f74bc-3644-4b98-a88b-1b345ffd53a8 req-641e731a-7413-4a0c-a847-d4d30b6b4f10 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:21.897205 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-cb8f74bc-3644-4b98-a88b-1b345ffd53a8 req-641e731a-7413-4a0c-a847-d4d30b6b4f10 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:21.898028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-36985ee1-179f-4bee-83f0-b69bb4388cd3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1555 microversion: 2.1 time: 0.311312 Jul 05 10:15:21.898604 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 276/577] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:21 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1555 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:21.913939 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-cb8f74bc-3644-4b98-a88b-1b345ffd53a8 req-641e731a-7413-4a0c-a847-d4d30b6b4f10 service nova] Creating event network-vif-deleted:b00d9c04-1d84-4211-8344-23f678f0b868 for instance 86fe0729-f79b-4fee-a79a-d004a32cfa2d on hv-cinder-maste Jul 05 10:15:21.914809 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d0686cfe-740c-4b6c-9213-c8462f322fc2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:21.915492 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d0686cfe-740c-4b6c-9213-c8462f322fc2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:21.921877 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-cb8f74bc-3644-4b98-a88b-1b345ffd53a8 req-641e731a-7413-4a0c-a847-d4d30b6b4f10 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040256 Jul 05 10:15:21.921877 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 302/578] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:15:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:15:21.927416 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d0686cfe-740c-4b6c-9213-c8462f322fc2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:21.928491 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d0686cfe-740c-4b6c-9213-c8462f322fc2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:21.929585 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d0686cfe-740c-4b6c-9213-c8462f322fc2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:21.937052 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-95b98b95-cea0-4201-af91-811dbcc7b1b0 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:21.937212 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d0686cfe-740c-4b6c-9213-c8462f322fc2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:21.937745 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-95b98b95-cea0-4201-af91-811dbcc7b1b0 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:21.943948 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-95b98b95-cea0-4201-af91-811dbcc7b1b0 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:21.944423 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-95b98b95-cea0-4201-af91-811dbcc7b1b0 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:21.945631 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-95b98b95-cea0-4201-af91-811dbcc7b1b0 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:21.946418 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-95b98b95-cea0-4201-af91-811dbcc7b1b0 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:22.054116 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-95b98b95-cea0-4201-af91-811dbcc7b1b0 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:22.065497 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-95b98b95-cea0-4201-af91-811dbcc7b1b0 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 200 len: 1304 microversion: 2.1 time: 0.133840 Jul 05 10:15:22.066210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 303/579] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:21 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 1304 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:22.081571 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d0686cfe-740c-4b6c-9213-c8462f322fc2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.170460 Jul 05 10:15:22.082152 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 277/580] 10.222.0.20 () {58 vars in 1274 bytes} [Tue Jul 5 10:15:21 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7/os-interface => generated 284 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:15:23.082965 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5b0a0cce-28be-4915-a041-95a3c7895287 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:23.083544 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5b0a0cce-28be-4915-a041-95a3c7895287 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 86fe0729-f79b-4fee-a79a-d004a32cfa2d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:23.089881 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5b0a0cce-28be-4915-a041-95a3c7895287 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:23.090329 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5b0a0cce-28be-4915-a041-95a3c7895287 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:23.091070 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5b0a0cce-28be-4915-a041-95a3c7895287 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:23.091569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5b0a0cce-28be-4915-a041-95a3c7895287 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:23.104726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-5b0a0cce-28be-4915-a041-95a3c7895287 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] HTTP exception thrown: Instance 86fe0729-f79b-4fee-a79a-d004a32cfa2d could not be found. Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5b0a0cce-28be-4915-a041-95a3c7895287 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Request method failure captured: Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d HTTP/1.1 Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA7M_oPdYVSPNdTZ-qpP7pPM4mdOKNNAAJNkA0WcZXKFrrxSbTwHUKyOwyLRMWn4IiYhaoAwhUs71nJmjt6NtYF3HGSsag1W5y-BgLvmVGFzg9sGXe1GID0AA6xXfjDAHCXlWAbtEH7lnQEc_Cmv_a3dHBAMmSyTIw4Lqdmdi5r4W48 Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: accabbba722345d190b52446d1e7c2b7 Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1826132577 Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:15:23.106341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:15:23.107775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/accabbba722345d190b52446d1e7c2b7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_accabbba722345d190b52446d1e7c2b7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/accabbba722345d190b52446d1e7c2b7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/accabbba722345d190b52446d1e7c2b7"}]}] Jul 05 10:15:23.107775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-VolumesSnapshotTestJSON-1826132577 Jul 05 10:15:23.107775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: accabbba722345d190b52446d1e7c2b7 Jul 05 10:15:23.107775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1826132577 Jul 05 10:15:23.107775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-VolumesSnapshotTestJSON-1826132577-project Jul 05 10:15:23.107775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:15:23.107775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:15:23.107775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: df495d1fbab24736a2ba12e8a0e7d803 Jul 05 10:15:23.107775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-VolumesSnapshotTestJSON-1826132577-project Jul 05 10:15:23.107775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:15:23.107775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance 86fe0729-f79b-4fee-a79a-d004a32cfa2d could not be found. Jul 05 10:15:23.107775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': '86fe0729-f79b-4fee-a79a-d004a32cfa2d'} Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 86fe0729-f79b-4fee-a79a-d004a32cfa2d could not be found. Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:15:23.108567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 86fe0729-f79b-4fee-a79a-d004a32cfa2d could not be found. Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 86fe0729-f79b-4fee-a79a-d004a32cfa2d could not be found. Jul 05 10:15:23.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:15:23.110933 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5b0a0cce-28be-4915-a041-95a3c7895287 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Returning 404 to user: Instance 86fe0729-f79b-4fee-a79a-d004a32cfa2d could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:15:23.110933 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5b0a0cce-28be-4915-a041-95a3c7895287 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d" status: 404 len: 111 microversion: 2.1 time: 0.028583 Jul 05 10:15:23.110933 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 304/581] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:23 2022] GET /compute/v2.1/servers/86fe0729-f79b-4fee-a79a-d004a32cfa2d => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:15:23.939623 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-7bab0ed1-4bca-4324-8258-58690d793e1e req-cc8f12d2-b405-489a-8d04-dbc83560823d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a325da12-82e8-4964-a4db-4d38efa52dc7", "tag": "de7ed2e8-b690-41d8-936e-1c400d4b704d"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:23.948089 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-7bab0ed1-4bca-4324-8258-58690d793e1e req-cc8f12d2-b405-489a-8d04-dbc83560823d service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:23.948555 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-7bab0ed1-4bca-4324-8258-58690d793e1e req-cc8f12d2-b405-489a-8d04-dbc83560823d service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:23.960966 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-7bab0ed1-4bca-4324-8258-58690d793e1e req-cc8f12d2-b405-489a-8d04-dbc83560823d service nova] Creating event network-changed:de7ed2e8-b690-41d8-936e-1c400d4b704d for instance a325da12-82e8-4964-a4db-4d38efa52dc7 on hv-cinder-maste Jul 05 10:15:23.965994 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-7bab0ed1-4bca-4324-8258-58690d793e1e req-cc8f12d2-b405-489a-8d04-dbc83560823d service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030970 Jul 05 10:15:23.966622 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 278/582] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:15:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:15:35.018522 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-857030157", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "networks": [{"uuid": "11e0e390-3962-4c1a-9a29-d936175172f2"}]}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:35.025804 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=34,nova_api:UPDATE=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:35.026459 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=82,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:35.072677 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.network.neutron [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] validate_networks() for [('11e0e390-3962-4c1a-9a29-d936175172f2', None, None, None, None, None)] {{(pid=117813) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:15:35.456960 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Flavor limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:15:35.457379 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Image limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:15:35.457848 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Flavor pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:15:35.458256 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Image pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:15:35.458660 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:15:35.467949 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:35.468414 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:35.469466 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:35.469904 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:35.486075 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Getting quotas for project accabbba722345d190b52446d1e7c2b7. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:15:35.489993 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Getting quotas for user df495d1fbab24736a2ba12e8a0e7d803 and project accabbba722345d190b52446d1e7c2b7. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:15:35.500790 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Going to run 1 instances... {{(pid=117813) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:15:35.502984 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117813) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:15:35.546618 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-de742455-d6d9-4d32-a4ca-e1fce0b8d3b2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.531919 Jul 05 10:15:35.547255 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 305/583] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:15:35 2022] POST /compute/v2.1/servers => generated 380 bytes in 533 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:15:35.555084 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=29 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:35.555819 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=75 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:35.560609 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-563c87f7-9d48-4481-9c6e-11430af851d7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:35.561303 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-563c87f7-9d48-4481-9c6e-11430af851d7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:35.617264 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-563c87f7-9d48-4481-9c6e-11430af851d7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:35.624516 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-563c87f7-9d48-4481-9c6e-11430af851d7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1218 microversion: 2.1 time: 0.071086 Jul 05 10:15:35.625262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 279/584] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:35 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1218 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:15:36.642382 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-07f7e361-eb0e-45b4-8599-b424e8d33e25 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:36.643621 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-07f7e361-eb0e-45b4-8599-b424e8d33e25 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:36.650330 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-07f7e361-eb0e-45b4-8599-b424e8d33e25 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:36.650782 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-07f7e361-eb0e-45b4-8599-b424e8d33e25 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:36.651546 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-07f7e361-eb0e-45b4-8599-b424e8d33e25 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:36.651982 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-07f7e361-eb0e-45b4-8599-b424e8d33e25 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:36.716999 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-07f7e361-eb0e-45b4-8599-b424e8d33e25 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:36.725621 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-07f7e361-eb0e-45b4-8599-b424e8d33e25 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1278 microversion: 2.1 time: 0.088129 Jul 05 10:15:36.726418 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 306/585] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:36 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1278 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:37.743493 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-21201d51-41ba-459b-9809-8e28c3751803 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:37.743927 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-21201d51-41ba-459b-9809-8e28c3751803 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:37.751769 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-21201d51-41ba-459b-9809-8e28c3751803 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:37.752429 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-21201d51-41ba-459b-9809-8e28c3751803 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:37.753198 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-21201d51-41ba-459b-9809-8e28c3751803 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:37.753627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-21201d51-41ba-459b-9809-8e28c3751803 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:37.972781 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-21201d51-41ba-459b-9809-8e28c3751803 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:37.980261 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-21201d51-41ba-459b-9809-8e28c3751803 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1318 microversion: 2.1 time: 0.240991 Jul 05 10:15:37.980788 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 280/586] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:37 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1318 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:38.144470 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-3a732206-79f8-4d6e-8bd6-fffe61131918 req-a59ba936-dda4-4587-a80e-be88aa43b1e6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "448f3f09-4db0-422d-b67c-100a0d43bd6b", "tag": "60956b08-a3e2-4158-940a-cb1f98a98765"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:38.152536 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-3a732206-79f8-4d6e-8bd6-fffe61131918 req-a59ba936-dda4-4587-a80e-be88aa43b1e6 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:38.152976 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-3a732206-79f8-4d6e-8bd6-fffe61131918 req-a59ba936-dda4-4587-a80e-be88aa43b1e6 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:38.163880 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-3a732206-79f8-4d6e-8bd6-fffe61131918 req-a59ba936-dda4-4587-a80e-be88aa43b1e6 service nova] Creating event network-changed:60956b08-a3e2-4158-940a-cb1f98a98765 for instance 448f3f09-4db0-422d-b67c-100a0d43bd6b on hv-cinder-maste Jul 05 10:15:38.168946 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-3a732206-79f8-4d6e-8bd6-fffe61131918 req-a59ba936-dda4-4587-a80e-be88aa43b1e6 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.088296 Jul 05 10:15:38.169836 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 307/587] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:15:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:15:38.996665 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fb50514c-a812-431d-9fce-b525f31e7f7f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:38.997271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fb50514c-a812-431d-9fce-b525f31e7f7f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:39.003685 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb50514c-a812-431d-9fce-b525f31e7f7f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:39.004127 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb50514c-a812-431d-9fce-b525f31e7f7f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:39.005060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb50514c-a812-431d-9fce-b525f31e7f7f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:39.005499 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb50514c-a812-431d-9fce-b525f31e7f7f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:39.131229 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fb50514c-a812-431d-9fce-b525f31e7f7f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:39.138594 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fb50514c-a812-431d-9fce-b525f31e7f7f tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1318 microversion: 2.1 time: 0.145721 Jul 05 10:15:39.139662 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 281/588] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:38 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1318 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:40.156640 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-034edee4-e3e0-4762-9a5d-d30a4078a6d7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:40.157334 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-034edee4-e3e0-4762-9a5d-d30a4078a6d7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:40.163476 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-034edee4-e3e0-4762-9a5d-d30a4078a6d7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:40.163921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-034edee4-e3e0-4762-9a5d-d30a4078a6d7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:40.164717 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-034edee4-e3e0-4762-9a5d-d30a4078a6d7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:40.165156 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-034edee4-e3e0-4762-9a5d-d30a4078a6d7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:40.296046 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-034edee4-e3e0-4762-9a5d-d30a4078a6d7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:40.303383 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-034edee4-e3e0-4762-9a5d-d30a4078a6d7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1318 microversion: 2.1 time: 0.151904 Jul 05 10:15:40.303938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 308/589] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:40 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1318 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:41.319781 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b5745aa4-a53d-4f42-bc2a-98e2058a43fb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:41.320483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b5745aa4-a53d-4f42-bc2a-98e2058a43fb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:41.327058 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b5745aa4-a53d-4f42-bc2a-98e2058a43fb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:41.327652 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b5745aa4-a53d-4f42-bc2a-98e2058a43fb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:41.328466 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b5745aa4-a53d-4f42-bc2a-98e2058a43fb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:41.328901 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b5745aa4-a53d-4f42-bc2a-98e2058a43fb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:41.456017 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b5745aa4-a53d-4f42-bc2a-98e2058a43fb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:41.464422 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b5745aa4-a53d-4f42-bc2a-98e2058a43fb tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1318 microversion: 2.1 time: 0.148405 Jul 05 10:15:41.464991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 282/590] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:41 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1318 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:42.486959 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-80b84ca6-ec6b-475c-a8c4-08be40b212a9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:42.487598 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-80b84ca6-ec6b-475c-a8c4-08be40b212a9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:42.494083 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80b84ca6-ec6b-475c-a8c4-08be40b212a9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:42.494472 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80b84ca6-ec6b-475c-a8c4-08be40b212a9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:42.495230 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80b84ca6-ec6b-475c-a8c4-08be40b212a9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:42.495660 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-80b84ca6-ec6b-475c-a8c4-08be40b212a9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:42.621198 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-80b84ca6-ec6b-475c-a8c4-08be40b212a9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:42.628497 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-80b84ca6-ec6b-475c-a8c4-08be40b212a9 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1318 microversion: 2.1 time: 0.145939 Jul 05 10:15:42.629058 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 309/591] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:42 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1318 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:43.644930 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-96f152f3-4b3c-4885-8151-13e55d74c94e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:43.645534 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-96f152f3-4b3c-4885-8151-13e55d74c94e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:43.651920 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-96f152f3-4b3c-4885-8151-13e55d74c94e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:43.652310 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-96f152f3-4b3c-4885-8151-13e55d74c94e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:43.653095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-96f152f3-4b3c-4885-8151-13e55d74c94e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:43.653528 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-96f152f3-4b3c-4885-8151-13e55d74c94e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:43.782320 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-96f152f3-4b3c-4885-8151-13e55d74c94e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:43.789793 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-96f152f3-4b3c-4885-8151-13e55d74c94e tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1318 microversion: 2.1 time: 0.148623 Jul 05 10:15:43.790363 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 283/592] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:43 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1318 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:44.806959 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-726226fa-b2c8-41a5-9d6e-9c4a8ec54484 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:44.807563 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-726226fa-b2c8-41a5-9d6e-9c4a8ec54484 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:44.814589 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-726226fa-b2c8-41a5-9d6e-9c4a8ec54484 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:44.815035 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-726226fa-b2c8-41a5-9d6e-9c4a8ec54484 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:44.815791 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-726226fa-b2c8-41a5-9d6e-9c4a8ec54484 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:44.816224 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-726226fa-b2c8-41a5-9d6e-9c4a8ec54484 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:44.948754 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-726226fa-b2c8-41a5-9d6e-9c4a8ec54484 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:44.956770 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-726226fa-b2c8-41a5-9d6e-9c4a8ec54484 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1318 microversion: 2.1 time: 0.153617 Jul 05 10:15:44.957447 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 310/593] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:44 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1318 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:45.044410 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-52372b99-b448-4644-860e-cb506cbf74c4 req-81f24f8c-8de4-406f-b426-bcaf0e27a196 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "448f3f09-4db0-422d-b67c-100a0d43bd6b", "name": "network-vif-plugged", "status": "completed", "tag": "60956b08-a3e2-4158-940a-cb1f98a98765"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:45.053643 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-52372b99-b448-4644-860e-cb506cbf74c4 req-81f24f8c-8de4-406f-b426-bcaf0e27a196 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:45.054077 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-52372b99-b448-4644-860e-cb506cbf74c4 req-81f24f8c-8de4-406f-b426-bcaf0e27a196 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:45.067102 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-52372b99-b448-4644-860e-cb506cbf74c4 req-81f24f8c-8de4-406f-b426-bcaf0e27a196 service nova] Creating event network-vif-plugged:60956b08-a3e2-4158-940a-cb1f98a98765 for instance 448f3f09-4db0-422d-b67c-100a0d43bd6b on hv-cinder-maste Jul 05 10:15:45.072004 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-52372b99-b448-4644-860e-cb506cbf74c4 req-81f24f8c-8de4-406f-b426-bcaf0e27a196 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031265 Jul 05 10:15:45.072696 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 284/594] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:15:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:15:45.974464 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-97ce3122-d1d2-450b-af0a-fd1f8b042ead tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:45.975095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-97ce3122-d1d2-450b-af0a-fd1f8b042ead tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:45.977067 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:45.977957 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=6 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:45.985438 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-97ce3122-d1d2-450b-af0a-fd1f8b042ead tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:45.986017 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-97ce3122-d1d2-450b-af0a-fd1f8b042ead tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:45.987168 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-97ce3122-d1d2-450b-af0a-fd1f8b042ead tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:45.987822 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-97ce3122-d1d2-450b-af0a-fd1f8b042ead tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:46.117549 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-97ce3122-d1d2-450b-af0a-fd1f8b042ead tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:46.125949 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-97ce3122-d1d2-450b-af0a-fd1f8b042ead tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1318 microversion: 2.1 time: 0.155889 Jul 05 10:15:46.126509 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 311/595] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:45 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1318 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:15:47.138945 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:15:47.143622 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-4dd7ceda-e48b-47d1-bdb4-9f80e16e9e67 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:47.144308 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-4dd7ceda-e48b-47d1-bdb4-9f80e16e9e67 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:47.150684 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4dd7ceda-e48b-47d1-bdb4-9f80e16e9e67 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:47.151130 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4dd7ceda-e48b-47d1-bdb4-9f80e16e9e67 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:47.151878 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4dd7ceda-e48b-47d1-bdb4-9f80e16e9e67 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:47.152313 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4dd7ceda-e48b-47d1-bdb4-9f80e16e9e67 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:47.281516 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-4dd7ceda-e48b-47d1-bdb4-9f80e16e9e67 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:47.288938 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-4dd7ceda-e48b-47d1-bdb4-9f80e16e9e67 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1318 microversion: 2.1 time: 0.150987 Jul 05 10:15:47.289564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 285/596] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:47 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1318 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:48.304628 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-be61438d-94cc-4efa-beaa-fdf876bf5581 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:48.305234 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-be61438d-94cc-4efa-beaa-fdf876bf5581 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:48.311662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-be61438d-94cc-4efa-beaa-fdf876bf5581 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:48.312101 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-be61438d-94cc-4efa-beaa-fdf876bf5581 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:48.313541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-be61438d-94cc-4efa-beaa-fdf876bf5581 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:48.313972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-be61438d-94cc-4efa-beaa-fdf876bf5581 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:48.441579 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-be61438d-94cc-4efa-beaa-fdf876bf5581 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:48.448832 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-be61438d-94cc-4efa-beaa-fdf876bf5581 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1318 microversion: 2.1 time: 0.148026 Jul 05 10:15:48.449403 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 312/597] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:48 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1318 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:49.465266 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0ed4adaa-4e87-439b-83c6-08e0f855347d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:15:49.465876 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0ed4adaa-4e87-439b-83c6-08e0f855347d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:49.473291 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0ed4adaa-4e87-439b-83c6-08e0f855347d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:49.473738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0ed4adaa-4e87-439b-83c6-08e0f855347d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:49.474499 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0ed4adaa-4e87-439b-83c6-08e0f855347d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:49.474935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0ed4adaa-4e87-439b-83c6-08e0f855347d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:49.600771 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-0ed4adaa-4e87-439b-83c6-08e0f855347d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:15:49.608145 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0ed4adaa-4e87-439b-83c6-08e0f855347d tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1507 microversion: 2.1 time: 0.146703 Jul 05 10:15:49.608739 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 286/598] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:15:49 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1507 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:15:49.624240 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5a5210de-2bd3-4745-b0d1-d21c78d381c7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2a819050-0fcd-4100-8548-2f92b6201e46", "device": "/dev/sdb"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:15:49.625830 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5a5210de-2bd3-4745-b0d1-d21c78d381c7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:15:49.632253 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5a5210de-2bd3-4745-b0d1-d21c78d381c7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:49.632740 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5a5210de-2bd3-4745-b0d1-d21c78d381c7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:49.633502 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5a5210de-2bd3-4745-b0d1-d21c78d381c7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:15:49.633940 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5a5210de-2bd3-4745-b0d1-d21c78d381c7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:15:49.876791 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5a5210de-2bd3-4745-b0d1-d21c78d381c7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "POST /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.255841 Jul 05 10:15:49.877388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 313/599] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:15:49 2022] POST /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b/os-volume_attachments => generated 194 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:16:02.413638 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:16:02.414339 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Tue Jul 5 10:16:02 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:16:13.238562 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-249becd4-c246-4d9d-b581-3bec9b45f0c2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:13.239038 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-249becd4-c246-4d9d-b581-3bec9b45f0c2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:13.241221 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=18 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:16:13.241858 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=38 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:16:13.247642 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-249becd4-c246-4d9d-b581-3bec9b45f0c2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:13.248085 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-249becd4-c246-4d9d-b581-3bec9b45f0c2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:13.248873 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-249becd4-c246-4d9d-b581-3bec9b45f0c2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:13.249307 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-249becd4-c246-4d9d-b581-3bec9b45f0c2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:13.368738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-249becd4-c246-4d9d-b581-3bec9b45f0c2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lazy-loading 'services' on Instance uuid 448f3f09-4db0-422d-b67c-100a0d43bd6b {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:16:13.479781 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-249becd4-c246-4d9d-b581-3bec9b45f0c2 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "DELETE /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b/os-volume_attachments/2a819050-0fcd-4100-8548-2f92b6201e46" status: 202 len: 0 microversion: 2.1 time: 0.245561 Jul 05 10:16:13.480381 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 287/600] 10.222.0.20 () {58 vars in 1415 bytes} [Tue Jul 5 10:16:13 2022] DELETE /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b/os-volume_attachments/2a819050-0fcd-4100-8548-2f92b6201e46 => generated 0 bytes in 247 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:16:15.644276 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=3 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:16:15.645073 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=51,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:16:15.649627 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8a49b1ce-52fe-45de-aac4-df55813e4103 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:15.650206 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8a49b1ce-52fe-45de-aac4-df55813e4103 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:15.657988 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8a49b1ce-52fe-45de-aac4-df55813e4103 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:15.658436 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8a49b1ce-52fe-45de-aac4-df55813e4103 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:15.659197 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8a49b1ce-52fe-45de-aac4-df55813e4103 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:15.659633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8a49b1ce-52fe-45de-aac4-df55813e4103 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:15.686585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8a49b1ce-52fe-45de-aac4-df55813e4103 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Going to try to terminate instance {{(pid=117813) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:16:15.760418 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8a49b1ce-52fe-45de-aac4-df55813e4103 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "DELETE /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 204 len: 0 microversion: 2.1 time: 0.117388 Jul 05 10:16:15.761958 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 314/601] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:16:15 2022] DELETE /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:16:15.770713 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-176c15a5-ea7a-4406-b65b-76f6e8bea758 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:15.771407 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-176c15a5-ea7a-4406-b65b-76f6e8bea758 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:15.778129 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-176c15a5-ea7a-4406-b65b-76f6e8bea758 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:15.778583 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-176c15a5-ea7a-4406-b65b-76f6e8bea758 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:15.779343 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-176c15a5-ea7a-4406-b65b-76f6e8bea758 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:15.779784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-176c15a5-ea7a-4406-b65b-76f6e8bea758 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:15.907544 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-176c15a5-ea7a-4406-b65b-76f6e8bea758 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:15.915624 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-176c15a5-ea7a-4406-b65b-76f6e8bea758 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1513 microversion: 2.1 time: 0.149789 Jul 05 10:16:15.916222 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 288/602] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:15 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1513 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:16:16.933107 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3871c2fc-ced6-465d-a89d-c9efd371cc13 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:16.933796 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3871c2fc-ced6-465d-a89d-c9efd371cc13 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:16.940274 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3871c2fc-ced6-465d-a89d-c9efd371cc13 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:16.940770 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3871c2fc-ced6-465d-a89d-c9efd371cc13 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:16.941528 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3871c2fc-ced6-465d-a89d-c9efd371cc13 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:16.941965 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3871c2fc-ced6-465d-a89d-c9efd371cc13 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:17.071924 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-3871c2fc-ced6-465d-a89d-c9efd371cc13 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:17.079311 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3871c2fc-ced6-465d-a89d-c9efd371cc13 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1513 microversion: 2.1 time: 0.150377 Jul 05 10:16:17.079866 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 315/603] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:16 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1513 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:18.096965 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9e88638b-a9e7-4cb0-87d8-36310b976654 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:18.097930 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9e88638b-a9e7-4cb0-87d8-36310b976654 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:18.105151 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9e88638b-a9e7-4cb0-87d8-36310b976654 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:18.105606 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9e88638b-a9e7-4cb0-87d8-36310b976654 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:18.106421 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9e88638b-a9e7-4cb0-87d8-36310b976654 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:18.106838 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9e88638b-a9e7-4cb0-87d8-36310b976654 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:18.183980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2ec66943-66b0-4763-92b8-bbff0826317a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a2c26058-b0da-463f-bc49-c3777e966958"}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:16:18.185220 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2ec66943-66b0-4763-92b8-bbff0826317a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:18.191276 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2ec66943-66b0-4763-92b8-bbff0826317a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:18.191827 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2ec66943-66b0-4763-92b8-bbff0826317a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:18.192694 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2ec66943-66b0-4763-92b8-bbff0826317a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:18.193211 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2ec66943-66b0-4763-92b8-bbff0826317a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:18.350429 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9e88638b-a9e7-4cb0-87d8-36310b976654 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:18.357788 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9e88638b-a9e7-4cb0-87d8-36310b976654 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1513 microversion: 2.1 time: 0.264996 Jul 05 10:16:18.358349 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 316/604] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:18 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1513 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:16:18.481461 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2ec66943-66b0-4763-92b8-bbff0826317a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "POST /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.403192 Jul 05 10:16:18.482008 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 289/605] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:16:18 2022] POST /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments => generated 194 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:16:18.768491 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-173d78f6-8937-4cf3-a70c-00aa660c4973 req-f6f5ae20-b579-45fe-8fbe-fdffdd4fd526 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "448f3f09-4db0-422d-b67c-100a0d43bd6b", "name": "network-vif-deleted", "tag": "60956b08-a3e2-4158-940a-cb1f98a98765"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:16:18.777897 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-173d78f6-8937-4cf3-a70c-00aa660c4973 req-f6f5ae20-b579-45fe-8fbe-fdffdd4fd526 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:18.778330 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-173d78f6-8937-4cf3-a70c-00aa660c4973 req-f6f5ae20-b579-45fe-8fbe-fdffdd4fd526 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:18.789203 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-173d78f6-8937-4cf3-a70c-00aa660c4973 req-f6f5ae20-b579-45fe-8fbe-fdffdd4fd526 service nova] Creating event network-vif-deleted:60956b08-a3e2-4158-940a-cb1f98a98765 for instance 448f3f09-4db0-422d-b67c-100a0d43bd6b on hv-cinder-maste Jul 05 10:16:18.794582 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-173d78f6-8937-4cf3-a70c-00aa660c4973 req-f6f5ae20-b579-45fe-8fbe-fdffdd4fd526 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030185 Jul 05 10:16:18.795244 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 317/606] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:16:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:16:19.375665 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-1d348897-dde6-4135-88e2-16a15173bbf7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:19.376408 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-1d348897-dde6-4135-88e2-16a15173bbf7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:19.382930 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d348897-dde6-4135-88e2-16a15173bbf7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:19.383383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d348897-dde6-4135-88e2-16a15173bbf7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:19.384289 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d348897-dde6-4135-88e2-16a15173bbf7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:19.384765 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d348897-dde6-4135-88e2-16a15173bbf7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:19.416884 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d348897-dde6-4135-88e2-16a15173bbf7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:19.417328 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d348897-dde6-4135-88e2-16a15173bbf7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:19.417770 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-1d348897-dde6-4135-88e2-16a15173bbf7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lazy-loading 'fault' on Instance uuid 448f3f09-4db0-422d-b67c-100a0d43bd6b {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:16:19.458815 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1d348897-dde6-4135-88e2-16a15173bbf7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'df495d1fbab24736a2ba12e8a0e7d803', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'accabbba722345d190b52446d1e7c2b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:19.466051 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-1d348897-dde6-4135-88e2-16a15173bbf7 tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 200 len: 1308 microversion: 2.1 time: 0.095068 Jul 05 10:16:19.466617 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 290/607] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:19 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 1308 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:20.484503 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-dd92e026-96c7-4103-ba63-50afa33630fe tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:20.485195 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-dd92e026-96c7-4103-ba63-50afa33630fe tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] [instance: 448f3f09-4db0-422d-b67c-100a0d43bd6b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:20.491472 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dd92e026-96c7-4103-ba63-50afa33630fe tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:20.491913 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dd92e026-96c7-4103-ba63-50afa33630fe tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:20.492731 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dd92e026-96c7-4103-ba63-50afa33630fe tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:20.493765 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dd92e026-96c7-4103-ba63-50afa33630fe tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:20.506843 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-dd92e026-96c7-4103-ba63-50afa33630fe tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] HTTP exception thrown: Instance 448f3f09-4db0-422d-b67c-100a0d43bd6b could not be found. Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-dd92e026-96c7-4103-ba63-50afa33630fe tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Request method failure captured: Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b HTTP/1.1 Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA7M_oPdYVSPNdTZ-qpP7pPM4mdOKNNAAJNkA0WcZXKFrrxSbTwHUKyOwyLRMWn4IiYhaoAwhUs71nJmjt6NtYF3HGSsag1W5y-BgLvmVGFzg9sGXe1GID0AA6xXfjDAHCXlWAbtEH7lnQEc_Cmv_a3dHBAMmSyTIw4Lqdmdi5r4W48 Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: accabbba722345d190b52446d1e7c2b7 Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1826132577 Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:16:20.508499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:16:20.510082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/accabbba722345d190b52446d1e7c2b7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_accabbba722345d190b52446d1e7c2b7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/accabbba722345d190b52446d1e7c2b7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/accabbba722345d190b52446d1e7c2b7"}]}] Jul 05 10:16:20.510082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-VolumesSnapshotTestJSON-1826132577 Jul 05 10:16:20.510082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: accabbba722345d190b52446d1e7c2b7 Jul 05 10:16:20.510082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1826132577 Jul 05 10:16:20.510082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-VolumesSnapshotTestJSON-1826132577-project Jul 05 10:16:20.510082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:16:20.510082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:16:20.510082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: df495d1fbab24736a2ba12e8a0e7d803 Jul 05 10:16:20.510082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-VolumesSnapshotTestJSON-1826132577-project Jul 05 10:16:20.510082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:16:20.510082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance 448f3f09-4db0-422d-b67c-100a0d43bd6b could not be found. Jul 05 10:16:20.510082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': '448f3f09-4db0-422d-b67c-100a0d43bd6b'} Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 448f3f09-4db0-422d-b67c-100a0d43bd6b could not be found. Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:16:20.510860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 448f3f09-4db0-422d-b67c-100a0d43bd6b could not be found. Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 448f3f09-4db0-422d-b67c-100a0d43bd6b could not be found. Jul 05 10:16:20.512118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:16:20.513581 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-dd92e026-96c7-4103-ba63-50afa33630fe tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] Returning 404 to user: Instance 448f3f09-4db0-422d-b67c-100a0d43bd6b could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:16:20.513581 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-dd92e026-96c7-4103-ba63-50afa33630fe tempest-VolumesSnapshotTestJSON-1826132577 tempest-VolumesSnapshotTestJSON-1826132577-project] 10.222.0.20 "GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b" status: 404 len: 111 microversion: 2.1 time: 0.030047 Jul 05 10:16:20.513581 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 318/608] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:20 2022] GET /compute/v2.1/servers/448f3f09-4db0-422d-b67c-100a0d43bd6b => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:16:28.070027 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-44e4f11b-8647-40ef-9cda-698ae4286873 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:16:28.071067 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-44e4f11b-8647-40ef-9cda-698ae4286873 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:28.078751 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-44e4f11b-8647-40ef-9cda-698ae4286873 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:28.079199 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-44e4f11b-8647-40ef-9cda-698ae4286873 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:28.079958 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-44e4f11b-8647-40ef-9cda-698ae4286873 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:28.080406 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-44e4f11b-8647-40ef-9cda-698ae4286873 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:28.103364 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-44e4f11b-8647-40ef-9cda-698ae4286873 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lazy-loading 'flavor' on Instance uuid 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:16:28.222945 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-44e4f11b-8647-40ef-9cda-698ae4286873 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "POST /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/action" status: 200 len: 32 microversion: 2.1 time: 0.156759 Jul 05 10:16:28.223862 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 291/609] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:16:28 2022] POST /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/action => generated 32 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 05 10:16:28.233011 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-92743964-6275-4030-a8a6-dc6973dbb5f6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:28.233715 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-92743964-6275-4030-a8a6-dc6973dbb5f6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:28.240298 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-92743964-6275-4030-a8a6-dc6973dbb5f6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:28.240819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-92743964-6275-4030-a8a6-dc6973dbb5f6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:28.241582 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-92743964-6275-4030-a8a6-dc6973dbb5f6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:28.242134 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-92743964-6275-4030-a8a6-dc6973dbb5f6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:28.707369 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-92743964-6275-4030-a8a6-dc6973dbb5f6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:28.715212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-92743964-6275-4030-a8a6-dc6973dbb5f6 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.485955 Jul 05 10:16:28.715776 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 319/610] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:28 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:29.733665 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-94de3733-7b97-4f41-a48e-7f88c0583f6e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:29.734466 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-94de3733-7b97-4f41-a48e-7f88c0583f6e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:29.740598 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-94de3733-7b97-4f41-a48e-7f88c0583f6e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:29.741048 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-94de3733-7b97-4f41-a48e-7f88c0583f6e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:29.741816 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-94de3733-7b97-4f41-a48e-7f88c0583f6e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:29.742257 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-94de3733-7b97-4f41-a48e-7f88c0583f6e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:29.883531 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-94de3733-7b97-4f41-a48e-7f88c0583f6e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:29.892246 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-94de3733-7b97-4f41-a48e-7f88c0583f6e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.163216 Jul 05 10:16:29.892852 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 292/611] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:29 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:30.908962 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-beaba544-4cee-4f78-b9c3-f183b7fbceef tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:30.909609 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-beaba544-4cee-4f78-b9c3-f183b7fbceef tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:30.916118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-beaba544-4cee-4f78-b9c3-f183b7fbceef tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:30.916603 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-beaba544-4cee-4f78-b9c3-f183b7fbceef tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:30.917357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-beaba544-4cee-4f78-b9c3-f183b7fbceef tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:30.917792 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-beaba544-4cee-4f78-b9c3-f183b7fbceef tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:31.062520 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-beaba544-4cee-4f78-b9c3-f183b7fbceef tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:31.070332 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-beaba544-4cee-4f78-b9c3-f183b7fbceef tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.165314 Jul 05 10:16:31.070896 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 320/612] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:30 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:32.090578 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0cdf0e5f-cece-43d1-a453-f5a4bc011c63 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:32.091194 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0cdf0e5f-cece-43d1-a453-f5a4bc011c63 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:32.097627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0cdf0e5f-cece-43d1-a453-f5a4bc011c63 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:32.098075 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0cdf0e5f-cece-43d1-a453-f5a4bc011c63 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:32.099082 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0cdf0e5f-cece-43d1-a453-f5a4bc011c63 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:32.099518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0cdf0e5f-cece-43d1-a453-f5a4bc011c63 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:32.225590 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-0cdf0e5f-cece-43d1-a453-f5a4bc011c63 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:32.234066 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0cdf0e5f-cece-43d1-a453-f5a4bc011c63 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.147838 Jul 05 10:16:32.234654 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 293/613] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:32 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:33.251238 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6358552f-f29d-4956-9ce7-01e4b97520da tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:33.251833 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-6358552f-f29d-4956-9ce7-01e4b97520da tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:33.258226 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6358552f-f29d-4956-9ce7-01e4b97520da tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:33.258672 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6358552f-f29d-4956-9ce7-01e4b97520da tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:33.259421 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6358552f-f29d-4956-9ce7-01e4b97520da tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:33.259852 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6358552f-f29d-4956-9ce7-01e4b97520da tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:33.390398 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6358552f-f29d-4956-9ce7-01e4b97520da tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:33.398185 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-6358552f-f29d-4956-9ce7-01e4b97520da tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.150765 Jul 05 10:16:33.398737 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 321/614] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:33 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:34.414956 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2ba48de5-7a91-4447-884a-5fe9b65d7b43 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:34.415569 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2ba48de5-7a91-4447-884a-5fe9b65d7b43 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:34.422015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2ba48de5-7a91-4447-884a-5fe9b65d7b43 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:34.422463 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2ba48de5-7a91-4447-884a-5fe9b65d7b43 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:34.423217 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2ba48de5-7a91-4447-884a-5fe9b65d7b43 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:34.423685 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2ba48de5-7a91-4447-884a-5fe9b65d7b43 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:34.549083 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-2ba48de5-7a91-4447-884a-5fe9b65d7b43 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:34.556985 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2ba48de5-7a91-4447-884a-5fe9b65d7b43 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.145850 Jul 05 10:16:34.558057 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 294/615] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:34 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:35.575026 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9b66b36f-1cbd-4ebf-878d-c34f8fdbd7f1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:35.575565 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9b66b36f-1cbd-4ebf-878d-c34f8fdbd7f1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:35.581930 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9b66b36f-1cbd-4ebf-878d-c34f8fdbd7f1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:35.582371 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9b66b36f-1cbd-4ebf-878d-c34f8fdbd7f1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:35.583115 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9b66b36f-1cbd-4ebf-878d-c34f8fdbd7f1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:35.583543 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9b66b36f-1cbd-4ebf-878d-c34f8fdbd7f1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:35.708025 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9b66b36f-1cbd-4ebf-878d-c34f8fdbd7f1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:35.716376 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9b66b36f-1cbd-4ebf-878d-c34f8fdbd7f1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.145725 Jul 05 10:16:35.716953 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 322/616] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:35 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:36.733602 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-177fb618-ae68-43b8-8b09-839ad649d2a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:36.734236 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-177fb618-ae68-43b8-8b09-839ad649d2a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:36.740569 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-177fb618-ae68-43b8-8b09-839ad649d2a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:36.741019 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-177fb618-ae68-43b8-8b09-839ad649d2a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:36.741779 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-177fb618-ae68-43b8-8b09-839ad649d2a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:36.742215 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-177fb618-ae68-43b8-8b09-839ad649d2a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:36.877219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-177fb618-ae68-43b8-8b09-839ad649d2a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:36.885091 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-177fb618-ae68-43b8-8b09-839ad649d2a3 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.155396 Jul 05 10:16:36.886213 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 295/617] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:36 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:16:37.903147 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b62c884c-50be-46d2-a6f6-1ac79bba99bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:37.903668 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b62c884c-50be-46d2-a6f6-1ac79bba99bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:37.910063 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b62c884c-50be-46d2-a6f6-1ac79bba99bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:37.910504 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b62c884c-50be-46d2-a6f6-1ac79bba99bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:37.911252 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b62c884c-50be-46d2-a6f6-1ac79bba99bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:37.911678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b62c884c-50be-46d2-a6f6-1ac79bba99bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:38.040488 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b62c884c-50be-46d2-a6f6-1ac79bba99bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:38.049098 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b62c884c-50be-46d2-a6f6-1ac79bba99bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.150530 Jul 05 10:16:38.049673 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 323/618] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:37 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:39.067557 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-35959c23-6f25-4846-8af5-e75fc9ff3a17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:39.068135 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-35959c23-6f25-4846-8af5-e75fc9ff3a17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:39.075359 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-35959c23-6f25-4846-8af5-e75fc9ff3a17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:39.075814 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-35959c23-6f25-4846-8af5-e75fc9ff3a17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:39.076626 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-35959c23-6f25-4846-8af5-e75fc9ff3a17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:39.077116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-35959c23-6f25-4846-8af5-e75fc9ff3a17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:39.206676 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-35959c23-6f25-4846-8af5-e75fc9ff3a17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:39.214555 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-35959c23-6f25-4846-8af5-e75fc9ff3a17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.150945 Jul 05 10:16:39.215130 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 296/619] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:39 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:40.231252 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-766295f2-6df6-4370-beff-0416d087fcbf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:40.231864 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-766295f2-6df6-4370-beff-0416d087fcbf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:40.238320 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-766295f2-6df6-4370-beff-0416d087fcbf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:40.238771 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-766295f2-6df6-4370-beff-0416d087fcbf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:40.239530 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-766295f2-6df6-4370-beff-0416d087fcbf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:40.239960 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-766295f2-6df6-4370-beff-0416d087fcbf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:40.372614 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-766295f2-6df6-4370-beff-0416d087fcbf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:40.381019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-766295f2-6df6-4370-beff-0416d087fcbf tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.153616 Jul 05 10:16:40.381584 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 324/620] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:40 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:41.398673 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-22eefdc7-c193-4f64-b05d-8a3f39027bff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:41.399277 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-22eefdc7-c193-4f64-b05d-8a3f39027bff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:41.406460 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22eefdc7-c193-4f64-b05d-8a3f39027bff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:41.406907 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22eefdc7-c193-4f64-b05d-8a3f39027bff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:41.407668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22eefdc7-c193-4f64-b05d-8a3f39027bff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:41.408179 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22eefdc7-c193-4f64-b05d-8a3f39027bff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:41.545527 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-22eefdc7-c193-4f64-b05d-8a3f39027bff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:41.553292 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-22eefdc7-c193-4f64-b05d-8a3f39027bff tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.158502 Jul 05 10:16:41.553851 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 297/621] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:41 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:42.570140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-18e65cb1-bea1-4f7b-bf9f-0ad3f8fa3f51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:42.570755 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-18e65cb1-bea1-4f7b-bf9f-0ad3f8fa3f51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:42.577207 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18e65cb1-bea1-4f7b-bf9f-0ad3f8fa3f51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:42.577658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18e65cb1-bea1-4f7b-bf9f-0ad3f8fa3f51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:42.578418 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18e65cb1-bea1-4f7b-bf9f-0ad3f8fa3f51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:42.578887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18e65cb1-bea1-4f7b-bf9f-0ad3f8fa3f51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:42.708149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-18e65cb1-bea1-4f7b-bf9f-0ad3f8fa3f51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:42.716561 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-18e65cb1-bea1-4f7b-bf9f-0ad3f8fa3f51 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.150241 Jul 05 10:16:42.717126 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 325/622] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:42 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:43.734626 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b4c34ccd-233b-4c33-8d12-e7eb4b91df3f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:43.735226 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b4c34ccd-233b-4c33-8d12-e7eb4b91df3f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:43.737950 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=25 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:16:43.743540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b4c34ccd-233b-4c33-8d12-e7eb4b91df3f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:43.744053 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b4c34ccd-233b-4c33-8d12-e7eb4b91df3f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:43.744993 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b4c34ccd-233b-4c33-8d12-e7eb4b91df3f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:43.745482 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b4c34ccd-233b-4c33-8d12-e7eb4b91df3f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:43.750845 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=86,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:16:43.880186 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b4c34ccd-233b-4c33-8d12-e7eb4b91df3f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:43.888000 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b4c34ccd-233b-4c33-8d12-e7eb4b91df3f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.158407 Jul 05 10:16:43.888587 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 298/623] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:43 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:44.904682 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e9dbe001-2b60-4e6e-a0ec-9a8d42731763 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:44.905286 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e9dbe001-2b60-4e6e-a0ec-9a8d42731763 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:44.911723 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e9dbe001-2b60-4e6e-a0ec-9a8d42731763 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:44.912166 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e9dbe001-2b60-4e6e-a0ec-9a8d42731763 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:44.912959 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e9dbe001-2b60-4e6e-a0ec-9a8d42731763 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:44.913392 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e9dbe001-2b60-4e6e-a0ec-9a8d42731763 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:45.051551 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e9dbe001-2b60-4e6e-a0ec-9a8d42731763 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:45.059479 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e9dbe001-2b60-4e6e-a0ec-9a8d42731763 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.158794 Jul 05 10:16:45.060043 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 326/624] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:44 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:46.077542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-36cdd515-4bde-4c8e-9cb0-502f6d42cc0f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:46.078177 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-36cdd515-4bde-4c8e-9cb0-502f6d42cc0f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:46.084739 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36cdd515-4bde-4c8e-9cb0-502f6d42cc0f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:46.085209 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36cdd515-4bde-4c8e-9cb0-502f6d42cc0f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:46.086022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36cdd515-4bde-4c8e-9cb0-502f6d42cc0f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:46.086464 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36cdd515-4bde-4c8e-9cb0-502f6d42cc0f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:46.231647 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-36cdd515-4bde-4c8e-9cb0-502f6d42cc0f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:46.240862 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-36cdd515-4bde-4c8e-9cb0-502f6d42cc0f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.167311 Jul 05 10:16:46.241360 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 299/625] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:46 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:47.257912 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-30361647-8827-4450-a9cd-386f405bb2d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:47.258514 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-30361647-8827-4450-a9cd-386f405bb2d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:47.260926 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=32,nova_api:UPDATE=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:16:47.266573 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-30361647-8827-4450-a9cd-386f405bb2d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:47.267018 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-30361647-8827-4450-a9cd-386f405bb2d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:47.267771 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-30361647-8827-4450-a9cd-386f405bb2d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:47.268202 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-30361647-8827-4450-a9cd-386f405bb2d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:47.272571 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=79,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:16:47.401276 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-30361647-8827-4450-a9cd-386f405bb2d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:47.409712 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-30361647-8827-4450-a9cd-386f405bb2d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1735 microversion: 2.1 time: 0.155713 Jul 05 10:16:47.410271 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 327/626] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:47 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1735 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:48.427516 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-290f3a87-7b4f-4ba2-9368-b7e8a5a5e045 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:48.428117 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-290f3a87-7b4f-4ba2-9368-b7e8a5a5e045 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:48.434549 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-290f3a87-7b4f-4ba2-9368-b7e8a5a5e045 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:48.434992 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-290f3a87-7b4f-4ba2-9368-b7e8a5a5e045 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:48.435745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-290f3a87-7b4f-4ba2-9368-b7e8a5a5e045 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:48.436178 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-290f3a87-7b4f-4ba2-9368-b7e8a5a5e045 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:48.561171 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-290f3a87-7b4f-4ba2-9368-b7e8a5a5e045 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:48.568927 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-290f3a87-7b4f-4ba2-9368-b7e8a5a5e045 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1715 microversion: 2.1 time: 0.145244 Jul 05 10:16:48.569481 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 300/627] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:48 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1715 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:48.672666 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d300cc1b-a14b-4abb-b48f-e7eb41c1e05f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:48.673171 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d300cc1b-a14b-4abb-b48f-e7eb41c1e05f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:48.679528 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d300cc1b-a14b-4abb-b48f-e7eb41c1e05f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:48.679970 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d300cc1b-a14b-4abb-b48f-e7eb41c1e05f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:48.680768 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d300cc1b-a14b-4abb-b48f-e7eb41c1e05f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:48.681197 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d300cc1b-a14b-4abb-b48f-e7eb41c1e05f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:48.759609 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-d300cc1b-a14b-4abb-b48f-e7eb41c1e05f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] HTTP exception thrown: Cannot 'detach_volume' instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd while it is in vm_state rescued Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d300cc1b-a14b-4abb-b48f-e7eb41c1e05f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Request method failure captured: Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: DELETE /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments/a2c26058-b0da-463f-bc49-c3777e966958 HTTP/1.1 Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4LMwEWQwEVIGHW57tb-mCZDddObx4iwQyZ5gk_llO51wkTkuVjOLNqddatV2FzjtkzWxyWr7Rmd5kPAnAKyMQJw43XAdu8pkSZT9OsjCAGydI1FCbBCPFs_pXG35EW9p3Z8AlqJqZdFJIfQwAG269sW3HEsdORJuVOabGGWKLHWOI Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 3f2199fa0bc54901aa24195a250a9e25 Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:16:48.761114 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:16:48.762676 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/3f2199fa0bc54901aa24195a250a9e25"}]}] Jul 05 10:16:48.762676 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:16:48.762676 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 3f2199fa0bc54901aa24195a250a9e25 Jul 05 10:16:48.762676 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:16:48.762676 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-ServerRescueNegativeTestJSON-1125341258-project Jul 05 10:16:48.762676 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:16:48.762676 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:16:48.762676 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 248946c73cb245f9b8e6fe73e46133cf Jul 05 10:16:48.762676 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1125341258-project Jul 05 10:16:48.762676 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:16:48.762676 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Cannot 'detach_volume' instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd while it is in vm_state rescued Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'server_id': '12e7fd00-8f3e-4fc4-882b-6ec193de0bfd', 'id': 'a2c26058-b0da-463f-bc49-c3777e966958'} Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd in vm_state rescued. Cannot detach_volume while the instance is in this state. Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:16:48.763374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd while it is in vm_state rescued Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd while it is in vm_state rescued Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:16:48.764726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d300cc1b-a14b-4abb-b48f-e7eb41c1e05f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Returning 409 to user: Cannot 'detach_volume' instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd while it is in vm_state rescued {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:16:48.766300 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d300cc1b-a14b-4abb-b48f-e7eb41c1e05f tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "DELETE /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments/a2c26058-b0da-463f-bc49-c3777e966958" status: 409 len: 152 microversion: 2.1 time: 0.095373 Jul 05 10:16:48.766300 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 328/628] 10.222.0.20 () {58 vars in 1415 bytes} [Tue Jul 5 10:16:48 2022] DELETE /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments/a2c26058-b0da-463f-bc49-c3777e966958 => generated 152 bytes in 97 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 05 10:16:48.777618 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-acf08897-c40c-49e9-bf82-df5342a18df9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:16:48.778145 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-acf08897-c40c-49e9-bf82-df5342a18df9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:48.784285 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-acf08897-c40c-49e9-bf82-df5342a18df9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:48.784887 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-acf08897-c40c-49e9-bf82-df5342a18df9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:48.785741 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-acf08897-c40c-49e9-bf82-df5342a18df9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:48.786302 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-acf08897-c40c-49e9-bf82-df5342a18df9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:48.851058 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-acf08897-c40c-49e9-bf82-df5342a18df9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "POST /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/action" status: 202 len: 0 microversion: 2.1 time: 0.076524 Jul 05 10:16:48.851689 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 301/629] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:16:48 2022] POST /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/action => generated 0 bytes in 79 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:16:48.860589 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f9ed3533-fd84-4e46-bcc4-109cd173ae2a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:48.861201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f9ed3533-fd84-4e46-bcc4-109cd173ae2a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:48.868555 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f9ed3533-fd84-4e46-bcc4-109cd173ae2a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:48.869005 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f9ed3533-fd84-4e46-bcc4-109cd173ae2a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:48.870514 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f9ed3533-fd84-4e46-bcc4-109cd173ae2a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:48.870955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f9ed3533-fd84-4e46-bcc4-109cd173ae2a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:49.088957 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f9ed3533-fd84-4e46-bcc4-109cd173ae2a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:49.097288 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f9ed3533-fd84-4e46-bcc4-109cd173ae2a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1723 microversion: 2.1 time: 0.241358 Jul 05 10:16:49.097853 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 329/630] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:48 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1723 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:50.115544 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b77bf0fb-525e-462b-81f9-43fea7bcf59c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:50.116247 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b77bf0fb-525e-462b-81f9-43fea7bcf59c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:50.123068 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b77bf0fb-525e-462b-81f9-43fea7bcf59c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:50.123522 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b77bf0fb-525e-462b-81f9-43fea7bcf59c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:50.124830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b77bf0fb-525e-462b-81f9-43fea7bcf59c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:50.125276 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b77bf0fb-525e-462b-81f9-43fea7bcf59c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:50.253723 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b77bf0fb-525e-462b-81f9-43fea7bcf59c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:50.261578 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b77bf0fb-525e-462b-81f9-43fea7bcf59c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1723 microversion: 2.1 time: 0.150257 Jul 05 10:16:50.262143 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 302/631] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:50 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1723 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:51.279335 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1477a3ef-5d5d-4227-9129-c491261b248d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:51.279858 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1477a3ef-5d5d-4227-9129-c491261b248d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:51.287136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1477a3ef-5d5d-4227-9129-c491261b248d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:51.287588 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1477a3ef-5d5d-4227-9129-c491261b248d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:51.288473 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1477a3ef-5d5d-4227-9129-c491261b248d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:51.288814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1477a3ef-5d5d-4227-9129-c491261b248d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:51.414351 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1477a3ef-5d5d-4227-9129-c491261b248d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:51.422247 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1477a3ef-5d5d-4227-9129-c491261b248d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1723 microversion: 2.1 time: 0.146828 Jul 05 10:16:51.422810 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 330/632] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:51 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1723 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:52.439413 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a5584251-538b-4acf-818a-39debb91d4c5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:52.440038 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a5584251-538b-4acf-818a-39debb91d4c5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:52.446491 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5584251-538b-4acf-818a-39debb91d4c5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:52.446932 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5584251-538b-4acf-818a-39debb91d4c5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:52.447682 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5584251-538b-4acf-818a-39debb91d4c5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:52.448111 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5584251-538b-4acf-818a-39debb91d4c5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:52.580010 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a5584251-538b-4acf-818a-39debb91d4c5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:16:52.587954 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a5584251-538b-4acf-818a-39debb91d4c5 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1729 microversion: 2.1 time: 0.152898 Jul 05 10:16:52.588542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 303/633] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:16:52 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1729 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:16:52.604552 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-50451911-c671-4947-bdd4-1e874e3c8434 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:16:52.605076 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-50451911-c671-4947-bdd4-1e874e3c8434 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:16:52.611943 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-50451911-c671-4947-bdd4-1e874e3c8434 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:52.612384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-50451911-c671-4947-bdd4-1e874e3c8434 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:52.613165 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-50451911-c671-4947-bdd4-1e874e3c8434 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:52.613686 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-50451911-c671-4947-bdd4-1e874e3c8434 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:52.690128 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-50451911-c671-4947-bdd4-1e874e3c8434 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.089591 Jul 05 10:16:52.690663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 331/634] 10.222.0.20 () {58 vars in 1274 bytes} [Tue Jul 5 10:16:52 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-interface => generated 285 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:16:53.781258 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-6dd1863d-bd09-4725-ad3d-eea79004f6ae req-918cc5ee-5a5f-467b-bd0f-a88acd3aa292 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "12e7fd00-8f3e-4fc4-882b-6ec193de0bfd", "tag": "e27d9ee6-1685-4aec-bb2d-ae3b3206c610"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:16:53.789658 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-6dd1863d-bd09-4725-ad3d-eea79004f6ae req-918cc5ee-5a5f-467b-bd0f-a88acd3aa292 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:53.790178 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-6dd1863d-bd09-4725-ad3d-eea79004f6ae req-918cc5ee-5a5f-467b-bd0f-a88acd3aa292 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:53.801138 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-6dd1863d-bd09-4725-ad3d-eea79004f6ae req-918cc5ee-5a5f-467b-bd0f-a88acd3aa292 service nova] Creating event network-changed:e27d9ee6-1685-4aec-bb2d-ae3b3206c610 for instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd on hv-cinder-maste Jul 05 10:16:53.807662 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-6dd1863d-bd09-4725-ad3d-eea79004f6ae req-918cc5ee-5a5f-467b-bd0f-a88acd3aa292 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029960 Jul 05 10:16:53.808532 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 304/635] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:16:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:16:55.822198 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-ff3a9676-c28b-468a-8fc6-5acc33c52a4d req-8e0eaa3b-2d74-46b4-9351-7df00e809012 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "12e7fd00-8f3e-4fc4-882b-6ec193de0bfd", "tag": "e27d9ee6-1685-4aec-bb2d-ae3b3206c610"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:16:55.830696 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-ff3a9676-c28b-468a-8fc6-5acc33c52a4d req-8e0eaa3b-2d74-46b4-9351-7df00e809012 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:16:55.831136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-ff3a9676-c28b-468a-8fc6-5acc33c52a4d req-8e0eaa3b-2d74-46b4-9351-7df00e809012 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:16:55.841834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-ff3a9676-c28b-468a-8fc6-5acc33c52a4d req-8e0eaa3b-2d74-46b4-9351-7df00e809012 service nova] Creating event network-changed:e27d9ee6-1685-4aec-bb2d-ae3b3206c610 for instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd on hv-cinder-maste Jul 05 10:16:55.847337 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-ff3a9676-c28b-468a-8fc6-5acc33c52a4d req-8e0eaa3b-2d74-46b4-9351-7df00e809012 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029247 Jul 05 10:16:55.847905 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 332/636] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:16:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:17:11.672338 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=13 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:11.673137 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=41,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:11.677468 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-48cd4207-734f-43a0-a162-9359ff9d790a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:17:11.678518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-48cd4207-734f-43a0-a162-9359ff9d790a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:11.685770 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-48cd4207-734f-43a0-a162-9359ff9d790a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:11.686219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-48cd4207-734f-43a0-a162-9359ff9d790a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:11.686984 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-48cd4207-734f-43a0-a162-9359ff9d790a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:11.687421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-48cd4207-734f-43a0-a162-9359ff9d790a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:11.760007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-48cd4207-734f-43a0-a162-9359ff9d790a tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "POST /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/action" status: 202 len: 0 microversion: 2.1 time: 0.089028 Jul 05 10:17:11.760719 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 305/637] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:17:11 2022] POST /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/action => generated 0 bytes in 91 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:17:11.766805 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=11 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:11.767522 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=29 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:11.772549 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0428d06f-0cd5-4458-8155-41e597a2469d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:11.773323 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0428d06f-0cd5-4458-8155-41e597a2469d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:11.780795 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0428d06f-0cd5-4458-8155-41e597a2469d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:11.781243 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0428d06f-0cd5-4458-8155-41e597a2469d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:11.782010 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0428d06f-0cd5-4458-8155-41e597a2469d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:11.782450 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0428d06f-0cd5-4458-8155-41e597a2469d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:12.018760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0428d06f-0cd5-4458-8155-41e597a2469d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:12.026850 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0428d06f-0cd5-4458-8155-41e597a2469d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1750 microversion: 2.1 time: 0.261693 Jul 05 10:17:12.027410 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 333/638] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:11 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1750 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:13.044850 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-255e03b4-d868-43f5-8173-1e8a5956aedf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:13.045541 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-255e03b4-d868-43f5-8173-1e8a5956aedf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:13.051800 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-255e03b4-d868-43f5-8173-1e8a5956aedf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:13.052285 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-255e03b4-d868-43f5-8173-1e8a5956aedf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:13.053093 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-255e03b4-d868-43f5-8173-1e8a5956aedf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:13.053533 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-255e03b4-d868-43f5-8173-1e8a5956aedf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:13.197255 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-255e03b4-d868-43f5-8173-1e8a5956aedf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:13.205156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-255e03b4-d868-43f5-8173-1e8a5956aedf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1755 microversion: 2.1 time: 0.165034 Jul 05 10:17:13.205718 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 306/639] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:13 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1755 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:14.223432 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d7af9c29-58b6-47d8-8700-ad1f1aed2ffb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:14.224036 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d7af9c29-58b6-47d8-8700-ad1f1aed2ffb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:14.230825 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d7af9c29-58b6-47d8-8700-ad1f1aed2ffb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:14.231271 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d7af9c29-58b6-47d8-8700-ad1f1aed2ffb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:14.232027 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d7af9c29-58b6-47d8-8700-ad1f1aed2ffb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:14.232484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d7af9c29-58b6-47d8-8700-ad1f1aed2ffb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:14.371135 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d7af9c29-58b6-47d8-8700-ad1f1aed2ffb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:14.379021 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d7af9c29-58b6-47d8-8700-ad1f1aed2ffb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1755 microversion: 2.1 time: 0.160048 Jul 05 10:17:14.379583 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 334/640] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:14 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1755 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:15.394879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-bce0427f-2c67-4e67-9f78-ef146c287ddb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:15.395510 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bce0427f-2c67-4e67-9f78-ef146c287ddb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:15.401920 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bce0427f-2c67-4e67-9f78-ef146c287ddb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:15.402366 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bce0427f-2c67-4e67-9f78-ef146c287ddb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:15.403116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bce0427f-2c67-4e67-9f78-ef146c287ddb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:15.403547 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bce0427f-2c67-4e67-9f78-ef146c287ddb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:15.544782 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-bce0427f-2c67-4e67-9f78-ef146c287ddb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:15.553158 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-bce0427f-2c67-4e67-9f78-ef146c287ddb tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1755 microversion: 2.1 time: 0.162102 Jul 05 10:17:15.553711 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 307/641] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:15 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1755 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:16.570923 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-22bcfa21-d51a-420b-8a06-b54a2e5bffa2 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:16.571442 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-22bcfa21-d51a-420b-8a06-b54a2e5bffa2 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:16.577881 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-22bcfa21-d51a-420b-8a06-b54a2e5bffa2 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:16.578322 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-22bcfa21-d51a-420b-8a06-b54a2e5bffa2 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:16.579103 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-22bcfa21-d51a-420b-8a06-b54a2e5bffa2 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:16.579532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-22bcfa21-d51a-420b-8a06-b54a2e5bffa2 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:16.718282 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-22bcfa21-d51a-420b-8a06-b54a2e5bffa2 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:16.726139 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-22bcfa21-d51a-420b-8a06-b54a2e5bffa2 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1755 microversion: 2.1 time: 0.159819 Jul 05 10:17:16.726693 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 335/642] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:16 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1755 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:17.743009 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a85beca7-0a93-472e-a6be-3c844e247d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:17.743613 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a85beca7-0a93-472e-a6be-3c844e247d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:17.750063 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a85beca7-0a93-472e-a6be-3c844e247d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:17.750515 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a85beca7-0a93-472e-a6be-3c844e247d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:17.751272 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a85beca7-0a93-472e-a6be-3c844e247d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:17.751708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a85beca7-0a93-472e-a6be-3c844e247d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:17.903465 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a85beca7-0a93-472e-a6be-3c844e247d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:17.911433 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a85beca7-0a93-472e-a6be-3c844e247d83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1755 microversion: 2.1 time: 0.172181 Jul 05 10:17:17.911992 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 308/643] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:17 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1755 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:18.929406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e4ff54d7-a7bc-41ec-8a1a-3258a70db2ef tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:18.930012 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e4ff54d7-a7bc-41ec-8a1a-3258a70db2ef tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:18.936009 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e4ff54d7-a7bc-41ec-8a1a-3258a70db2ef tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:18.936487 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e4ff54d7-a7bc-41ec-8a1a-3258a70db2ef tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:18.937253 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e4ff54d7-a7bc-41ec-8a1a-3258a70db2ef tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:18.937689 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e4ff54d7-a7bc-41ec-8a1a-3258a70db2ef tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:19.077842 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e4ff54d7-a7bc-41ec-8a1a-3258a70db2ef tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:19.086209 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e4ff54d7-a7bc-41ec-8a1a-3258a70db2ef tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1755 microversion: 2.1 time: 0.161295 Jul 05 10:17:19.086771 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 336/644] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:18 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1755 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:20.103774 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f810bea9-f7d3-4097-8594-303ccba2bfd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:20.104382 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f810bea9-f7d3-4097-8594-303ccba2bfd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:20.110951 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f810bea9-f7d3-4097-8594-303ccba2bfd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:20.111396 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f810bea9-f7d3-4097-8594-303ccba2bfd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:20.112156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f810bea9-f7d3-4097-8594-303ccba2bfd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:20.112620 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f810bea9-f7d3-4097-8594-303ccba2bfd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:20.253155 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f810bea9-f7d3-4097-8594-303ccba2bfd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:20.261550 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f810bea9-f7d3-4097-8594-303ccba2bfd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1755 microversion: 2.1 time: 0.161862 Jul 05 10:17:20.262107 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 309/645] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:20 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1755 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:21.278364 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-7d456d1b-0588-437e-926d-7a6ac55e2a06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:21.278974 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7d456d1b-0588-437e-926d-7a6ac55e2a06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:21.285510 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7d456d1b-0588-437e-926d-7a6ac55e2a06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:21.285954 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7d456d1b-0588-437e-926d-7a6ac55e2a06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:21.287295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7d456d1b-0588-437e-926d-7a6ac55e2a06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:21.287736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7d456d1b-0588-437e-926d-7a6ac55e2a06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:21.426641 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-7d456d1b-0588-437e-926d-7a6ac55e2a06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:21.434449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-7d456d1b-0588-437e-926d-7a6ac55e2a06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1755 microversion: 2.1 time: 0.159905 Jul 05 10:17:21.435048 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 337/646] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:21 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1755 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:21.980548 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-205fccaa-06ec-4bbe-a24b-b1eba4115aea tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:21.981016 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-205fccaa-06ec-4bbe-a24b-b1eba4115aea tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:21.987554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-205fccaa-06ec-4bbe-a24b-b1eba4115aea tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:21.988003 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-205fccaa-06ec-4bbe-a24b-b1eba4115aea tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:21.988785 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-205fccaa-06ec-4bbe-a24b-b1eba4115aea tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:21.989721 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-205fccaa-06ec-4bbe-a24b-b1eba4115aea tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:22.012520 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-205fccaa-06ec-4bbe-a24b-b1eba4115aea tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7/ips" status: 200 len: 149 microversion: 2.1 time: 0.036084 Jul 05 10:17:22.013065 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 310/647] 10.222.0.20 () {58 vars in 1247 bytes} [Tue Jul 5 10:17:21 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7/ips => generated 149 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:17:22.023210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8ced237f-602c-4b19-8c8b-22e409b4220a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "fc1afd39-9160-4b8f-b920-a5c714959b0e"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:17:22.024332 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8ced237f-602c-4b19-8c8b-22e409b4220a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:22.030420 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ced237f-602c-4b19-8c8b-22e409b4220a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:22.030873 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ced237f-602c-4b19-8c8b-22e409b4220a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:22.031635 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ced237f-602c-4b19-8c8b-22e409b4220a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:22.032070 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ced237f-602c-4b19-8c8b-22e409b4220a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:22.108040 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-8ced237f-602c-4b19-8c8b-22e409b4220a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Invalid volume: volume fc1afd39-9160-4b8f-b920-a5c714959b0e is already attached to instances: ab80efc4-b2fa-4761-806b-2a197cc5b00b Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8ced237f-602c-4b19-8c8b-22e409b4220a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: POST /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7/os-volume_attachments HTTP/1.1 Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 74 Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:17:22.109829 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:17:22.111443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:17:22.111443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:17:22.111443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:17:22.111443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:17:22.111443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:17:22.111443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:17:22.111443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:17:22.111443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:17:22.111443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:17:22.111443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Jul 05 10:17:22.111443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {"volumeAttachment": {"volumeId": "fc1afd39-9160-4b8f-b920-a5c714959b0e"}} Jul 05 10:17:22.111443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Invalid volume: volume fc1afd39-9160-4b8f-b920-a5c714959b0e is already attached to instances: ab80efc4-b2fa-4761-806b-2a197cc5b00b Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'server_id': 'a325da12-82e8-4964-a4db-4d38efa52dc7', 'body': {'volumeAttachment': {'volumeId': 'fc1afd39-9160-4b8f-b920-a5c714959b0e'}}} Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume fc1afd39-9160-4b8f-b920-a5c714959b0e is already attached to instances: ab80efc4-b2fa-4761-806b-2a197cc5b00b Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:17:22.112449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume fc1afd39-9160-4b8f-b920-a5c714959b0e is already attached to instances: ab80efc4-b2fa-4761-806b-2a197cc5b00b Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:17:22.113955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:17:22.116009 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:17:22.116009 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:17:22.116009 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:17:22.116009 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume fc1afd39-9160-4b8f-b920-a5c714959b0e is already attached to instances: ab80efc4-b2fa-4761-806b-2a197cc5b00b Jul 05 10:17:22.116009 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:17:22.116009 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8ced237f-602c-4b19-8c8b-22e409b4220a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 400 to user: Invalid volume: volume fc1afd39-9160-4b8f-b920-a5c714959b0e is already attached to instances: ab80efc4-b2fa-4761-806b-2a197cc5b00b {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:17:22.116009 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8ced237f-602c-4b19-8c8b-22e409b4220a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "POST /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.092010 Jul 05 10:17:22.116009 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 338/648] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:17:22 2022] POST /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7/os-volume_attachments => generated 174 bytes in 93 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 05 10:17:22.120500 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-59d9cbcc-a156-48ee-97d0-8141e4185bf7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:22.120992 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-59d9cbcc-a156-48ee-97d0-8141e4185bf7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:22.126983 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59d9cbcc-a156-48ee-97d0-8141e4185bf7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:22.127428 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59d9cbcc-a156-48ee-97d0-8141e4185bf7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:22.128182 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59d9cbcc-a156-48ee-97d0-8141e4185bf7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:22.128654 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59d9cbcc-a156-48ee-97d0-8141e4185bf7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:22.154293 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-59d9cbcc-a156-48ee-97d0-8141e4185bf7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Going to try to terminate instance {{(pid=117812) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:17:22.226994 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-59d9cbcc-a156-48ee-97d0-8141e4185bf7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 204 len: 0 microversion: 2.1 time: 0.109831 Jul 05 10:17:22.227929 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 311/649] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:17:22 2022] DELETE /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 05 10:17:22.236845 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-73831a38-a797-4c0c-a574-91003782b93b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:22.237450 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-73831a38-a797-4c0c-a574-91003782b93b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:22.244003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-73831a38-a797-4c0c-a574-91003782b93b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:22.244474 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-73831a38-a797-4c0c-a574-91003782b93b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:22.245236 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-73831a38-a797-4c0c-a574-91003782b93b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:22.245667 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-73831a38-a797-4c0c-a574-91003782b93b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:22.375668 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-73831a38-a797-4c0c-a574-91003782b93b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:22.383086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-73831a38-a797-4c0c-a574-91003782b93b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1680 microversion: 2.1 time: 0.150708 Jul 05 10:17:22.383648 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 339/650] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:22 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1680 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:22.451637 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-39fc9f83-0463-4252-b09b-cc82ddb8b51f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:22.452328 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-39fc9f83-0463-4252-b09b-cc82ddb8b51f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:22.458782 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-39fc9f83-0463-4252-b09b-cc82ddb8b51f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:22.459229 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-39fc9f83-0463-4252-b09b-cc82ddb8b51f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:22.459991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-39fc9f83-0463-4252-b09b-cc82ddb8b51f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:22.460450 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-39fc9f83-0463-4252-b09b-cc82ddb8b51f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:22.601068 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-39fc9f83-0463-4252-b09b-cc82ddb8b51f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:22.609132 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-39fc9f83-0463-4252-b09b-cc82ddb8b51f tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1755 microversion: 2.1 time: 0.161677 Jul 05 10:17:22.609698 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 312/651] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:22 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1755 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:23.402163 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3b3b5dc9-71c8-426e-a540-7d62b8e315fb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:23.402769 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3b3b5dc9-71c8-426e-a540-7d62b8e315fb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:23.409236 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3b3b5dc9-71c8-426e-a540-7d62b8e315fb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:23.409682 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3b3b5dc9-71c8-426e-a540-7d62b8e315fb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:23.410437 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3b3b5dc9-71c8-426e-a540-7d62b8e315fb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:23.410906 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3b3b5dc9-71c8-426e-a540-7d62b8e315fb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:23.535678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-3b3b5dc9-71c8-426e-a540-7d62b8e315fb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:23.543566 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3b3b5dc9-71c8-426e-a540-7d62b8e315fb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1680 microversion: 2.1 time: 0.145288 Jul 05 10:17:23.544151 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 340/652] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:23 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1680 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:23.625758 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6a064b2e-53e8-45b0-b953-3d0a4ac09958 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:23.626363 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6a064b2e-53e8-45b0-b953-3d0a4ac09958 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:23.633346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6a064b2e-53e8-45b0-b953-3d0a4ac09958 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:23.633792 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6a064b2e-53e8-45b0-b953-3d0a4ac09958 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:23.634545 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6a064b2e-53e8-45b0-b953-3d0a4ac09958 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:23.634979 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6a064b2e-53e8-45b0-b953-3d0a4ac09958 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:23.776553 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6a064b2e-53e8-45b0-b953-3d0a4ac09958 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:23.784301 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6a064b2e-53e8-45b0-b953-3d0a4ac09958 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1755 microversion: 2.1 time: 0.162343 Jul 05 10:17:23.784886 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 313/653] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:23 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1755 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:24.560439 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f5231c84-da7b-44a3-a156-1859ee295363 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:24.561033 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f5231c84-da7b-44a3-a156-1859ee295363 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:24.567457 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f5231c84-da7b-44a3-a156-1859ee295363 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:24.567894 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f5231c84-da7b-44a3-a156-1859ee295363 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:24.568682 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f5231c84-da7b-44a3-a156-1859ee295363 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:24.569131 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f5231c84-da7b-44a3-a156-1859ee295363 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:24.719730 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f5231c84-da7b-44a3-a156-1859ee295363 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:24.728044 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f5231c84-da7b-44a3-a156-1859ee295363 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1680 microversion: 2.1 time: 0.171154 Jul 05 10:17:24.728653 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 341/654] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:24 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1680 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:24.801157 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f56760eb-fed3-4c2f-90de-87525675f8a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:24.801765 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f56760eb-fed3-4c2f-90de-87525675f8a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:24.808110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f56760eb-fed3-4c2f-90de-87525675f8a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:24.808588 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f56760eb-fed3-4c2f-90de-87525675f8a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:24.809356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f56760eb-fed3-4c2f-90de-87525675f8a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:24.809796 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f56760eb-fed3-4c2f-90de-87525675f8a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:25.048682 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f56760eb-fed3-4c2f-90de-87525675f8a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:25.056871 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f56760eb-fed3-4c2f-90de-87525675f8a5 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1758 microversion: 2.1 time: 0.259256 Jul 05 10:17:25.057447 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 314/655] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:24 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1758 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:25.744790 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2a8ac2ab-9423-4e03-8eda-6c08b3a1c742 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:25.746580 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2a8ac2ab-9423-4e03-8eda-6c08b3a1c742 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:25.754328 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2a8ac2ab-9423-4e03-8eda-6c08b3a1c742 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:25.755048 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2a8ac2ab-9423-4e03-8eda-6c08b3a1c742 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:25.756640 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2a8ac2ab-9423-4e03-8eda-6c08b3a1c742 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:25.757252 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2a8ac2ab-9423-4e03-8eda-6c08b3a1c742 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:25.899658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2a8ac2ab-9423-4e03-8eda-6c08b3a1c742 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:25.907742 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2a8ac2ab-9423-4e03-8eda-6c08b3a1c742 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 200 len: 1613 microversion: 2.1 time: 0.166666 Jul 05 10:17:25.917525 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 342/656] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:25 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 1613 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:25.921573 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-7b55d8ae-fd59-4e5d-ae8a-1dd3bfe637e8 req-b5190ca5-7ed5-442c-b19a-7f3c130b9f5d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a325da12-82e8-4964-a4db-4d38efa52dc7", "name": "network-vif-deleted", "tag": "de7ed2e8-b690-41d8-936e-1c400d4b704d"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:17:25.930943 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-7b55d8ae-fd59-4e5d-ae8a-1dd3bfe637e8 req-b5190ca5-7ed5-442c-b19a-7f3c130b9f5d service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:25.931390 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-7b55d8ae-fd59-4e5d-ae8a-1dd3bfe637e8 req-b5190ca5-7ed5-442c-b19a-7f3c130b9f5d service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:25.943241 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-7b55d8ae-fd59-4e5d-ae8a-1dd3bfe637e8 req-b5190ca5-7ed5-442c-b19a-7f3c130b9f5d service nova] Creating event network-vif-deleted:de7ed2e8-b690-41d8-936e-1c400d4b704d for instance a325da12-82e8-4964-a4db-4d38efa52dc7 on hv-cinder-maste Jul 05 10:17:25.948442 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-7b55d8ae-fd59-4e5d-ae8a-1dd3bfe637e8 req-b5190ca5-7ed5-442c-b19a-7f3c130b9f5d service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039061 Jul 05 10:17:25.949309 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 315/657] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:17:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:17:26.924589 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3cb12713-b899-4a03-9661-f30fa7621b25 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:26.925368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3cb12713-b899-4a03-9661-f30fa7621b25 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:26.933422 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3cb12713-b899-4a03-9661-f30fa7621b25 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:26.933855 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3cb12713-b899-4a03-9661-f30fa7621b25 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:26.934628 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3cb12713-b899-4a03-9661-f30fa7621b25 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:26.935061 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3cb12713-b899-4a03-9661-f30fa7621b25 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:26.950917 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-3cb12713-b899-4a03-9661-f30fa7621b25 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3cb12713-b899-4a03-9661-f30fa7621b25 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 HTTP/1.1 Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:17:26.952489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:17:26.953811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:17:26.953811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:17:26.953811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:17:26.953811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:17:26.953811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:17:26.953811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:17:26.953811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:17:26.953811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:17:26.953811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:17:26.953811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:17:26.953811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:17:26.953811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'a325da12-82e8-4964-a4db-4d38efa52dc7'} Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:17:26.955084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:17:26.956585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:17:26.958773 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3cb12713-b899-4a03-9661-f30fa7621b25 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 404 to user: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:17:26.958773 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3cb12713-b899-4a03-9661-f30fa7621b25 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 404 len: 111 microversion: 2.1 time: 0.033285 Jul 05 10:17:26.958773 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 343/658] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:26 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:17:26.964403 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-19eb703d-c3fd-4568-824f-10e23b162035 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '.version_select at 0x7feb89691d30>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:26.985490 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-19eb703d-c3fd-4568-824f-10e23b162035 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2076576628" status: 202 len: 0 microversion: 2.1 time: 0.025569 Jul 05 10:17:26.986085 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 316/659] 10.222.0.20 () {58 vars in 1220 bytes} [Tue Jul 5 10:17:26 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2076576628 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:17:27.651258 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-641e825c-c17b-475a-b962-d4d2eb3db681 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:27.651782 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-641e825c-c17b-475a-b962-d4d2eb3db681 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:27.657942 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-641e825c-c17b-475a-b962-d4d2eb3db681 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:27.658602 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-641e825c-c17b-475a-b962-d4d2eb3db681 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:27.659372 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-641e825c-c17b-475a-b962-d4d2eb3db681 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:27.659805 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-641e825c-c17b-475a-b962-d4d2eb3db681 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:27.741897 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.objects.instance [None req-641e825c-c17b-475a-b962-d4d2eb3db681 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lazy-loading 'services' on Instance uuid ab80efc4-b2fa-4761-806b-2a197cc5b00b {{(pid=117813) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:17:27.810876 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-641e825c-c17b-475a-b962-d4d2eb3db681 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b/os-volume_attachments/fc1afd39-9160-4b8f-b920-a5c714959b0e" status: 202 len: 0 microversion: 2.1 time: 0.163492 Jul 05 10:17:27.811452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 344/660] 10.222.0.20 () {58 vars in 1415 bytes} [Tue Jul 5 10:17:27 2022] DELETE /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b/os-volume_attachments/fc1afd39-9160-4b8f-b920-a5c714959b0e => generated 0 bytes in 165 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:17:27.821277 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6b5a4419-5031-45fb-ba3c-995a65bc74f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:27.822458 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6b5a4419-5031-45fb-ba3c-995a65bc74f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:27.830723 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6b5a4419-5031-45fb-ba3c-995a65bc74f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:27.831257 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6b5a4419-5031-45fb-ba3c-995a65bc74f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:27.832063 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6b5a4419-5031-45fb-ba3c-995a65bc74f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:27.832920 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6b5a4419-5031-45fb-ba3c-995a65bc74f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:27.863885 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6b5a4419-5031-45fb-ba3c-995a65bc74f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048011 Jul 05 10:17:27.864881 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 317/661] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:17:27 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:17:28.876462 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d975ed32-7a8f-4f17-a29d-eb1fc9e76dd0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:28.877544 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d975ed32-7a8f-4f17-a29d-eb1fc9e76dd0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:28.884240 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d975ed32-7a8f-4f17-a29d-eb1fc9e76dd0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:28.884722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d975ed32-7a8f-4f17-a29d-eb1fc9e76dd0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:28.885492 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d975ed32-7a8f-4f17-a29d-eb1fc9e76dd0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:28.885928 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d975ed32-7a8f-4f17-a29d-eb1fc9e76dd0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:28.916211 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d975ed32-7a8f-4f17-a29d-eb1fc9e76dd0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044636 Jul 05 10:17:28.916793 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 345/662] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:17:28 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 05 10:17:28.974586 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-387b0265-cf47-4d1c-8f96-88af4ddef11c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:28.975075 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-387b0265-cf47-4d1c-8f96-88af4ddef11c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:28.982123 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-387b0265-cf47-4d1c-8f96-88af4ddef11c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:28.982559 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-387b0265-cf47-4d1c-8f96-88af4ddef11c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:28.983311 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-387b0265-cf47-4d1c-8f96-88af4ddef11c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:28.983743 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-387b0265-cf47-4d1c-8f96-88af4ddef11c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:29.010867 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-387b0265-cf47-4d1c-8f96-88af4ddef11c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Going to try to terminate instance {{(pid=117812) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:17:29.089128 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-387b0265-cf47-4d1c-8f96-88af4ddef11c tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 204 len: 0 microversion: 2.1 time: 0.117959 Jul 05 10:17:29.089635 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 318/663] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:17:28 2022] DELETE /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:17:29.097586 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f3562ec9-886b-4c8e-8dbf-179de8e887a0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:29.098182 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f3562ec9-886b-4c8e-8dbf-179de8e887a0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:29.104258 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f3562ec9-886b-4c8e-8dbf-179de8e887a0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:29.104729 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f3562ec9-886b-4c8e-8dbf-179de8e887a0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:29.105478 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f3562ec9-886b-4c8e-8dbf-179de8e887a0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:29.105908 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f3562ec9-886b-4c8e-8dbf-179de8e887a0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:29.236537 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f3562ec9-886b-4c8e-8dbf-179de8e887a0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:29.244200 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f3562ec9-886b-4c8e-8dbf-179de8e887a0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1681 microversion: 2.1 time: 0.150421 Jul 05 10:17:29.244787 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 346/664] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:29 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1681 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:30.262456 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c9881524-ad2a-4f2d-b663-bc3bec30cde1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:30.263062 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-c9881524-ad2a-4f2d-b663-bc3bec30cde1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:30.269717 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c9881524-ad2a-4f2d-b663-bc3bec30cde1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:30.270160 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c9881524-ad2a-4f2d-b663-bc3bec30cde1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:30.270914 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c9881524-ad2a-4f2d-b663-bc3bec30cde1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:30.271351 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c9881524-ad2a-4f2d-b663-bc3bec30cde1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:30.275291 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:30.474884 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-c9881524-ad2a-4f2d-b663-bc3bec30cde1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:30.482549 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-c9881524-ad2a-4f2d-b663-bc3bec30cde1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1681 microversion: 2.1 time: 0.224340 Jul 05 10:17:30.483112 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 319/665] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:30 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1681 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:31.201603 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-e3bc1a9e-4dea-41a5-bdce-86fc5cbffa4b req-185f846c-a924-4cb1-9c89-a30ee5399786 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ab80efc4-b2fa-4761-806b-2a197cc5b00b", "name": "network-vif-deleted", "tag": "66542ef5-f0db-4a7d-ab6e-ceb382db45de"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:17:31.211137 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-e3bc1a9e-4dea-41a5-bdce-86fc5cbffa4b req-185f846c-a924-4cb1-9c89-a30ee5399786 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:31.212215 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-e3bc1a9e-4dea-41a5-bdce-86fc5cbffa4b req-185f846c-a924-4cb1-9c89-a30ee5399786 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:31.224459 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-e3bc1a9e-4dea-41a5-bdce-86fc5cbffa4b req-185f846c-a924-4cb1-9c89-a30ee5399786 service nova] Creating event network-vif-deleted:66542ef5-f0db-4a7d-ab6e-ceb382db45de for instance ab80efc4-b2fa-4761-806b-2a197cc5b00b on hv-cinder-maste Jul 05 10:17:31.230854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-e3bc1a9e-4dea-41a5-bdce-86fc5cbffa4b req-185f846c-a924-4cb1-9c89-a30ee5399786 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033795 Jul 05 10:17:31.231451 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 347/666] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:17:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:17:31.280881 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f6db0645-0f27-42c9-867a-2387643d7ba9 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:17:31.282140 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f6db0645-0f27-42c9-867a-2387643d7ba9 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:31.290079 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f6db0645-0f27-42c9-867a-2387643d7ba9 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:31.290989 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f6db0645-0f27-42c9-867a-2387643d7ba9 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:31.292128 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f6db0645-0f27-42c9-867a-2387643d7ba9 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:31.292948 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f6db0645-0f27-42c9-867a-2387643d7ba9 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:31.319236 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-f6db0645-0f27-42c9-867a-2387643d7ba9 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lazy-loading 'flavor' on Instance uuid 4684ec3d-0407-461f-8a19-d6d8f99ca49b {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:17:31.409577 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f6db0645-0f27-42c9-867a-2387643d7ba9 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "POST /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b/action" status: 202 len: 0 microversion: 2.1 time: 0.132667 Jul 05 10:17:31.410500 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 320/667] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:17:31 2022] POST /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b/action => generated 0 bytes in 135 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:17:31.419395 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1256bfcc-097a-4e2b-bb48-3052e81ea621 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:31.420256 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1256bfcc-097a-4e2b-bb48-3052e81ea621 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:31.428880 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1256bfcc-097a-4e2b-bb48-3052e81ea621 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:31.429332 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1256bfcc-097a-4e2b-bb48-3052e81ea621 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:31.430116 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1256bfcc-097a-4e2b-bb48-3052e81ea621 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:31.431001 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1256bfcc-097a-4e2b-bb48-3052e81ea621 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:31.502479 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-bb6007d2-9b0a-42c9-a048-0bc881f31f37 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:31.503092 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bb6007d2-9b0a-42c9-a048-0bc881f31f37 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:31.510272 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bb6007d2-9b0a-42c9-a048-0bc881f31f37 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:31.510719 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bb6007d2-9b0a-42c9-a048-0bc881f31f37 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:31.511499 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bb6007d2-9b0a-42c9-a048-0bc881f31f37 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:31.511935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bb6007d2-9b0a-42c9-a048-0bc881f31f37 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:31.545221 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bb6007d2-9b0a-42c9-a048-0bc881f31f37 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:31.545665 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bb6007d2-9b0a-42c9-a048-0bc881f31f37 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:31.546109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-bb6007d2-9b0a-42c9-a048-0bc881f31f37 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lazy-loading 'fault' on Instance uuid ab80efc4-b2fa-4761-806b-2a197cc5b00b {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:17:31.585813 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1256bfcc-097a-4e2b-bb48-3052e81ea621 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:31.593897 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1256bfcc-097a-4e2b-bb48-3052e81ea621 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.178788 Jul 05 10:17:31.601297 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-bb6007d2-9b0a-42c9-a048-0bc881f31f37 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:31.604057 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 348/668] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:31 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:17:31.608834 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-bb6007d2-9b0a-42c9-a048-0bc881f31f37 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 200 len: 1331 microversion: 2.1 time: 0.112163 Jul 05 10:17:31.609532 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 321/669] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:31 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 1331 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:17:32.611304 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e50f0069-7d1f-4bee-9488-d842598b5a16 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:32.611822 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e50f0069-7d1f-4bee-9488-d842598b5a16 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:32.618391 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e50f0069-7d1f-4bee-9488-d842598b5a16 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:32.618834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e50f0069-7d1f-4bee-9488-d842598b5a16 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:32.619579 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e50f0069-7d1f-4bee-9488-d842598b5a16 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:32.620008 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e50f0069-7d1f-4bee-9488-d842598b5a16 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:32.625144 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-da9ba9b9-3232-456f-9951-ed856095a34d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:32.625754 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-da9ba9b9-3232-456f-9951-ed856095a34d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:32.632807 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-da9ba9b9-3232-456f-9951-ed856095a34d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:32.633260 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-da9ba9b9-3232-456f-9951-ed856095a34d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:32.634023 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-da9ba9b9-3232-456f-9951-ed856095a34d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:32.634461 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-da9ba9b9-3232-456f-9951-ed856095a34d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:32.648115 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-da9ba9b9-3232-456f-9951-ed856095a34d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-da9ba9b9-3232-456f-9951-ed856095a34d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b HTTP/1.1 Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:17:32.649764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:17:32.651095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:17:32.651095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:17:32.651095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:17:32.651095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:17:32.651095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:17:32.651095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:17:32.651095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:17:32.651095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:17:32.651095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:17:32.651095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:17:32.651095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:17:32.651095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': 'ab80efc4-b2fa-4761-806b-2a197cc5b00b'} Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:17:32.652097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:17:32.653564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:17:32.655004 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-da9ba9b9-3232-456f-9951-ed856095a34d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 404 to user: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:17:32.655004 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-da9ba9b9-3232-456f-9951-ed856095a34d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 404 len: 111 microversion: 2.1 time: 0.029778 Jul 05 10:17:32.655004 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 322/670] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:32 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 05 10:17:32.659527 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2fe7e63f-82bd-4e75-a9bf-c9515d232fdd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '.version_select at 0x7feb8954c9d0>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:32.676783 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2fe7e63f-82bd-4e75-a9bf-c9515d232fdd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1853656315" status: 202 len: 0 microversion: 2.1 time: 0.020626 Jul 05 10:17:32.677475 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 323/671] 10.222.0.20 () {58 vars in 1220 bytes} [Tue Jul 5 10:17:32 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1853656315 => generated 0 bytes in 22 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 05 10:17:32.751859 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e50f0069-7d1f-4bee-9488-d842598b5a16 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:32.760122 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e50f0069-7d1f-4bee-9488-d842598b5a16 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.152641 Jul 05 10:17:32.760793 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 349/672] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:32 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:33.303222 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-adfec88b-3f2a-4a14-8aaf-d2f65359f6a0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Action: 'create', calling method: .version_select at 0x7feb895db4c0>, body: {"keypair": {"name": "tempest-keypair-973925554", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPC8ZXmU0oxJb7Dd/5XdSnFzFsa348OogJ9wr5tCCMItXkxyF0aVcTujiuvoAS+iS1kSBQDChbOV9MIFUAPseLwcZNdqFfDxmtJ5VLnU2Gayo48DW5TgFBPb95L928v3EQ=="}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:17:33.317701 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-adfec88b-3f2a-4a14-8aaf-d2f65359f6a0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:17:33.322062 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-adfec88b-3f2a-4a14-8aaf-d2f65359f6a0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for user 036cb887c7894e75ba3eb4c1a8b21c69 and project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:17:33.342840 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-adfec88b-3f2a-4a14-8aaf-d2f65359f6a0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.044023 Jul 05 10:17:33.343426 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 324/673] 10.222.0.20 () {60 vars in 1158 bytes} [Tue Jul 5 10:17:33 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:17:33.777690 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8ffe1c94-359c-47bc-ad84-fee4777ef72c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:33.778406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8ffe1c94-359c-47bc-ad84-fee4777ef72c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:33.784966 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ffe1c94-359c-47bc-ad84-fee4777ef72c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:33.785405 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ffe1c94-359c-47bc-ad84-fee4777ef72c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:33.786195 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ffe1c94-359c-47bc-ad84-fee4777ef72c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:33.787235 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ffe1c94-359c-47bc-ad84-fee4777ef72c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:33.791914 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:33.999651 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8ffe1c94-359c-47bc-ad84-fee4777ef72c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:34.007064 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8ffe1c94-359c-47bc-ad84-fee4777ef72c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.233259 Jul 05 10:17:34.007617 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 350/674] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:33 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:34.658223 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-393233351", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "033e3c55-aad3-40e5-8b8b-1dedb7b873c7"}], "security_groups": [{"name": "tempest-securitygroup--191017446"}], "key_name": "tempest-keypair-973925554", "user_data": "}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:17:34.755780 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.network.neutron [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] validate_networks() for [('033e3c55-aad3-40e5-8b8b-1dedb7b873c7', None, None, None, None, None)] {{(pid=117812) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:17:35.024755 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-dcac1c1b-679b-4855-9839-17b0b45133d1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:35.025362 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-dcac1c1b-679b-4855-9839-17b0b45133d1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:35.032631 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dcac1c1b-679b-4855-9839-17b0b45133d1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:35.033079 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dcac1c1b-679b-4855-9839-17b0b45133d1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:35.033836 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dcac1c1b-679b-4855-9839-17b0b45133d1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:35.034264 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dcac1c1b-679b-4855-9839-17b0b45133d1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:35.252815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-dcac1c1b-679b-4855-9839-17b0b45133d1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:35.260824 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-dcac1c1b-679b-4855-9839-17b0b45133d1 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.240393 Jul 05 10:17:35.261393 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 351/675] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:35 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:17:35.278469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Flavor limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:17:35.278897 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Image limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:17:35.279373 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Flavor pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:17:35.279788 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Image pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:17:35.280200 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:17:35.289027 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:35.289485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:35.290291 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:35.290732 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:35.307916 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:17:35.311872 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for user 036cb887c7894e75ba3eb4c1a8b21c69 and project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:17:35.323310 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Going to run 1 instances... {{(pid=117812) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:17:35.325574 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117812) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:17:35.371042 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-49c6851d-e402-4715-b864-fa25b2c4bc41 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.717137 Jul 05 10:17:35.371683 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 325/676] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:17:34 2022] POST /compute/v2.1/servers => generated 401 bytes in 718 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:17:35.382038 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-38898c72-9c2d-4610-a41c-54c3899e2e65 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:35.382692 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-38898c72-9c2d-4610-a41c-54c3899e2e65 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:35.439971 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-38898c72-9c2d-4610-a41c-54c3899e2e65 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:35.447334 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-38898c72-9c2d-4610-a41c-54c3899e2e65 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 200 len: 1240 microversion: 2.1 time: 0.069669 Jul 05 10:17:35.447894 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 352/677] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:35 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 1240 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:36.278936 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-59b4b2bd-b9af-4681-bdc6-c18e51a1cb19 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:36.279615 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-59b4b2bd-b9af-4681-bdc6-c18e51a1cb19 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:36.285690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59b4b2bd-b9af-4681-bdc6-c18e51a1cb19 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:36.286144 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59b4b2bd-b9af-4681-bdc6-c18e51a1cb19 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:36.287519 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59b4b2bd-b9af-4681-bdc6-c18e51a1cb19 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:36.287960 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59b4b2bd-b9af-4681-bdc6-c18e51a1cb19 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:36.464900 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0cb955d9-26fc-48f9-b017-21297ad0edb1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:36.465520 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0cb955d9-26fc-48f9-b017-21297ad0edb1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:36.479860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0cb955d9-26fc-48f9-b017-21297ad0edb1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:36.480224 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0cb955d9-26fc-48f9-b017-21297ad0edb1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:36.481909 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0cb955d9-26fc-48f9-b017-21297ad0edb1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:36.482354 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0cb955d9-26fc-48f9-b017-21297ad0edb1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:36.514139 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-59b4b2bd-b9af-4681-bdc6-c18e51a1cb19 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:36.522905 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-59b4b2bd-b9af-4681-bdc6-c18e51a1cb19 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.248571 Jul 05 10:17:36.523614 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 326/678] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:36 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:36.562964 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0cb955d9-26fc-48f9-b017-21297ad0edb1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:36.570511 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0cb955d9-26fc-48f9-b017-21297ad0edb1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 200 len: 1298 microversion: 2.1 time: 0.109541 Jul 05 10:17:36.571068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 353/679] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:36 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 1298 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:37.539303 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6b5637d5-65c5-49bc-aa47-077320234959 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:37.540042 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6b5637d5-65c5-49bc-aa47-077320234959 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:37.546819 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6b5637d5-65c5-49bc-aa47-077320234959 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:37.547287 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6b5637d5-65c5-49bc-aa47-077320234959 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:37.548077 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6b5637d5-65c5-49bc-aa47-077320234959 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:37.548548 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6b5637d5-65c5-49bc-aa47-077320234959 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:37.587312 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-7df99cbd-fd5f-4037-8df3-0bf64bc37258 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:37.587951 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7df99cbd-fd5f-4037-8df3-0bf64bc37258 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:37.594457 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7df99cbd-fd5f-4037-8df3-0bf64bc37258 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:37.594915 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7df99cbd-fd5f-4037-8df3-0bf64bc37258 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:37.595688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7df99cbd-fd5f-4037-8df3-0bf64bc37258 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:37.596123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7df99cbd-fd5f-4037-8df3-0bf64bc37258 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:37.861540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6b5637d5-65c5-49bc-aa47-077320234959 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:37.869500 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6b5637d5-65c5-49bc-aa47-077320234959 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.334137 Jul 05 10:17:37.870084 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 327/680] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:37 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:37.912648 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-7df99cbd-fd5f-4037-8df3-0bf64bc37258 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:37.920014 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-7df99cbd-fd5f-4037-8df3-0bf64bc37258 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 200 len: 1365 microversion: 2.1 time: 0.336677 Jul 05 10:17:37.920873 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 354/681] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:37 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 1365 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:17:38.282356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-57ddf2fd-33ae-4dd5-bf21-07d266737a59 req-3f6b3a40-4dfa-467b-a6b7-8176a71810e2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "70cdd8c1-3058-4809-a0d1-73e39cf2e520", "tag": "dbd17c56-2b84-42e7-9867-b2fcb6dc597e"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:17:38.295117 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-57ddf2fd-33ae-4dd5-bf21-07d266737a59 req-3f6b3a40-4dfa-467b-a6b7-8176a71810e2 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:38.295564 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-57ddf2fd-33ae-4dd5-bf21-07d266737a59 req-3f6b3a40-4dfa-467b-a6b7-8176a71810e2 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:38.308539 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-57ddf2fd-33ae-4dd5-bf21-07d266737a59 req-3f6b3a40-4dfa-467b-a6b7-8176a71810e2 service nova] Creating event network-changed:dbd17c56-2b84-42e7-9867-b2fcb6dc597e for instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 on hv-cinder-maste Jul 05 10:17:38.313721 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-57ddf2fd-33ae-4dd5-bf21-07d266737a59 req-3f6b3a40-4dfa-467b-a6b7-8176a71810e2 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036686 Jul 05 10:17:38.315741 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 328/682] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:17:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:17:38.886160 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-675732f7-893a-4b53-89ed-3df914199382 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:38.886752 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-675732f7-893a-4b53-89ed-3df914199382 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:38.893213 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-675732f7-893a-4b53-89ed-3df914199382 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:38.893688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-675732f7-893a-4b53-89ed-3df914199382 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:38.894444 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-675732f7-893a-4b53-89ed-3df914199382 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:38.894883 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-675732f7-893a-4b53-89ed-3df914199382 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:38.937405 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-204cb263-3dda-4ca4-b47a-65f79d8f824a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:38.938102 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-204cb263-3dda-4ca4-b47a-65f79d8f824a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:38.944785 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-204cb263-3dda-4ca4-b47a-65f79d8f824a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:38.945242 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-204cb263-3dda-4ca4-b47a-65f79d8f824a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:38.946015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-204cb263-3dda-4ca4-b47a-65f79d8f824a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:38.946457 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-204cb263-3dda-4ca4-b47a-65f79d8f824a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:39.079248 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-675732f7-893a-4b53-89ed-3df914199382 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:39.087211 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-675732f7-893a-4b53-89ed-3df914199382 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.204916 Jul 05 10:17:39.087781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 355/683] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:38 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:39.109786 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-204cb263-3dda-4ca4-b47a-65f79d8f824a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:39.117577 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-204cb263-3dda-4ca4-b47a-65f79d8f824a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 200 len: 1365 microversion: 2.1 time: 0.184331 Jul 05 10:17:39.118136 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 329/684] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:38 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 1365 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:17:40.104735 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1b474c42-9f67-404d-857c-eb6654000b33 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:40.105337 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1b474c42-9f67-404d-857c-eb6654000b33 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:40.111695 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1b474c42-9f67-404d-857c-eb6654000b33 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:40.112139 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1b474c42-9f67-404d-857c-eb6654000b33 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:40.112975 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1b474c42-9f67-404d-857c-eb6654000b33 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:40.113407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1b474c42-9f67-404d-857c-eb6654000b33 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:40.134055 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7c26ffaf-b358-4026-85b5-28aaa40dd7db tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:40.134663 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7c26ffaf-b358-4026-85b5-28aaa40dd7db tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:40.141046 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7c26ffaf-b358-4026-85b5-28aaa40dd7db tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:40.141500 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7c26ffaf-b358-4026-85b5-28aaa40dd7db tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:40.142269 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7c26ffaf-b358-4026-85b5-28aaa40dd7db tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:40.142707 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7c26ffaf-b358-4026-85b5-28aaa40dd7db tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:40.299282 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1b474c42-9f67-404d-857c-eb6654000b33 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:40.307423 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1b474c42-9f67-404d-857c-eb6654000b33 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.206938 Jul 05 10:17:40.308285 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 356/685] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:40 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:40.321073 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-7c26ffaf-b358-4026-85b5-28aaa40dd7db tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:40.328383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7c26ffaf-b358-4026-85b5-28aaa40dd7db tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 200 len: 1365 microversion: 2.1 time: 0.198173 Jul 05 10:17:40.328955 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 330/686] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:40 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 1365 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:17:41.346739 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-213dcb50-3acc-462b-81d5-a784c2c5cf3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:41.349718 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-213dcb50-3acc-462b-81d5-a784c2c5cf3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:41.361893 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-213dcb50-3acc-462b-81d5-a784c2c5cf3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:41.362455 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-213dcb50-3acc-462b-81d5-a784c2c5cf3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:41.363332 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-213dcb50-3acc-462b-81d5-a784c2c5cf3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:41.363864 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-213dcb50-3acc-462b-81d5-a784c2c5cf3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:41.523950 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5095e94b-3395-4099-ad72-1909f7b61924 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:41.524619 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5095e94b-3395-4099-ad72-1909f7b61924 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:41.532192 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5095e94b-3395-4099-ad72-1909f7b61924 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:41.532775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5095e94b-3395-4099-ad72-1909f7b61924 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:41.533749 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5095e94b-3395-4099-ad72-1909f7b61924 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:41.534179 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5095e94b-3395-4099-ad72-1909f7b61924 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:41.594213 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-213dcb50-3acc-462b-81d5-a784c2c5cf3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:41.602298 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-213dcb50-3acc-462b-81d5-a784c2c5cf3a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 200 len: 1365 microversion: 2.1 time: 0.260898 Jul 05 10:17:41.603398 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 331/687] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:41 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 1365 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:41.753882 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5095e94b-3395-4099-ad72-1909f7b61924 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:41.761472 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5095e94b-3395-4099-ad72-1909f7b61924 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.440777 Jul 05 10:17:41.762035 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 357/688] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:41 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:42.619176 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-952ae43c-7fc2-4df5-b487-a28bc6314ebb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:42.619786 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-952ae43c-7fc2-4df5-b487-a28bc6314ebb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:42.622177 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=91,nova_api:UPDATE=2,nova_api:DELETE=2,nova_api:INSERT=4 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:42.627614 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-952ae43c-7fc2-4df5-b487-a28bc6314ebb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:42.628084 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-952ae43c-7fc2-4df5-b487-a28bc6314ebb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:42.628867 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-952ae43c-7fc2-4df5-b487-a28bc6314ebb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:42.629301 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-952ae43c-7fc2-4df5-b487-a28bc6314ebb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:42.758952 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-952ae43c-7fc2-4df5-b487-a28bc6314ebb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:42.766224 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-952ae43c-7fc2-4df5-b487-a28bc6314ebb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 200 len: 1365 microversion: 2.1 time: 0.150853 Jul 05 10:17:42.766783 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 332/689] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:42 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 1365 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:42.778695 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-84229bf0-80a7-4444-a65b-699c76f1beeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:42.779299 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-84229bf0-80a7-4444-a65b-699c76f1beeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:42.781711 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=53 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:42.787041 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-84229bf0-80a7-4444-a65b-699c76f1beeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:42.787484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-84229bf0-80a7-4444-a65b-699c76f1beeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:42.788237 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-84229bf0-80a7-4444-a65b-699c76f1beeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:42.788698 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-84229bf0-80a7-4444-a65b-699c76f1beeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:42.917941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-84229bf0-80a7-4444-a65b-699c76f1beeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:42.925516 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-84229bf0-80a7-4444-a65b-699c76f1beeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.150616 Jul 05 10:17:42.926084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 358/690] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:42 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:43.524951 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-fe3d79ea-bc47-4d25-90ca-eb93953d3003 req-266faca7-cb47-4c4c-a42c-29170afa4f61 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "70cdd8c1-3058-4809-a0d1-73e39cf2e520", "name": "network-vif-plugged", "status": "completed", "tag": "dbd17c56-2b84-42e7-9867-b2fcb6dc597e"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:17:43.532917 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-fe3d79ea-bc47-4d25-90ca-eb93953d3003 req-266faca7-cb47-4c4c-a42c-29170afa4f61 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:43.533354 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-fe3d79ea-bc47-4d25-90ca-eb93953d3003 req-266faca7-cb47-4c4c-a42c-29170afa4f61 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:43.546347 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-fe3d79ea-bc47-4d25-90ca-eb93953d3003 req-266faca7-cb47-4c4c-a42c-29170afa4f61 service nova] Creating event network-vif-plugged:dbd17c56-2b84-42e7-9867-b2fcb6dc597e for instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 on hv-cinder-maste Jul 05 10:17:43.551114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-fe3d79ea-bc47-4d25-90ca-eb93953d3003 req-266faca7-cb47-4c4c-a42c-29170afa4f61 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091971 Jul 05 10:17:43.551683 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 333/691] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:17:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:17:43.783168 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b86d7e46-b2ac-43dd-9615-1f5d968afd7a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:43.783771 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b86d7e46-b2ac-43dd-9615-1f5d968afd7a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:43.790959 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b86d7e46-b2ac-43dd-9615-1f5d968afd7a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:43.791403 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b86d7e46-b2ac-43dd-9615-1f5d968afd7a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:43.792166 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b86d7e46-b2ac-43dd-9615-1f5d968afd7a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:43.792625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b86d7e46-b2ac-43dd-9615-1f5d968afd7a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:43.922863 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b86d7e46-b2ac-43dd-9615-1f5d968afd7a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:43.930745 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b86d7e46-b2ac-43dd-9615-1f5d968afd7a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 200 len: 1365 microversion: 2.1 time: 0.151364 Jul 05 10:17:43.931444 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 359/692] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:43 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 1365 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:43.945333 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a8696edd-084c-4f53-bba6-649893266d46 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:43.946026 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a8696edd-084c-4f53-bba6-649893266d46 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:43.952074 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a8696edd-084c-4f53-bba6-649893266d46 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:43.952562 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a8696edd-084c-4f53-bba6-649893266d46 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:43.953318 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a8696edd-084c-4f53-bba6-649893266d46 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:43.953752 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a8696edd-084c-4f53-bba6-649893266d46 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:44.084581 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a8696edd-084c-4f53-bba6-649893266d46 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:44.091955 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a8696edd-084c-4f53-bba6-649893266d46 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.151745 Jul 05 10:17:44.092544 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 334/693] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:43 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:44.948115 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-4a77079e-0129-4853-837d-6a293c1bc6d7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:44.949169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-4a77079e-0129-4853-837d-6a293c1bc6d7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:44.955831 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4a77079e-0129-4853-837d-6a293c1bc6d7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:44.956286 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4a77079e-0129-4853-837d-6a293c1bc6d7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:44.957078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4a77079e-0129-4853-837d-6a293c1bc6d7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:44.957511 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4a77079e-0129-4853-837d-6a293c1bc6d7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:45.085996 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4a77079e-0129-4853-837d-6a293c1bc6d7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:45.093297 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-4a77079e-0129-4853-837d-6a293c1bc6d7 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 200 len: 1365 microversion: 2.1 time: 0.149977 Jul 05 10:17:45.093885 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 360/694] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:44 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 1365 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:45.109528 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-637270cc-f2ba-4f97-a300-8424ba22dbdc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:45.110663 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-637270cc-f2ba-4f97-a300-8424ba22dbdc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:45.119714 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-637270cc-f2ba-4f97-a300-8424ba22dbdc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:45.120153 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-637270cc-f2ba-4f97-a300-8424ba22dbdc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:45.120948 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-637270cc-f2ba-4f97-a300-8424ba22dbdc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:45.121383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-637270cc-f2ba-4f97-a300-8424ba22dbdc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:45.254131 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-637270cc-f2ba-4f97-a300-8424ba22dbdc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:45.262496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-637270cc-f2ba-4f97-a300-8424ba22dbdc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.158076 Jul 05 10:17:45.263192 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 335/695] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:45 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:46.110060 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-03a6bcd4-a97f-4fb3-b9f9-ae1bd0180985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:46.110574 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-03a6bcd4-a97f-4fb3-b9f9-ae1bd0180985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:46.112385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:46.118257 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-03a6bcd4-a97f-4fb3-b9f9-ae1bd0180985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:46.118709 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-03a6bcd4-a97f-4fb3-b9f9-ae1bd0180985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:46.119470 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-03a6bcd4-a97f-4fb3-b9f9-ae1bd0180985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:46.119909 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-03a6bcd4-a97f-4fb3-b9f9-ae1bd0180985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:46.245066 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-03a6bcd4-a97f-4fb3-b9f9-ae1bd0180985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:46.252601 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-03a6bcd4-a97f-4fb3-b9f9-ae1bd0180985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 200 len: 1554 microversion: 2.1 time: 0.146412 Jul 05 10:17:46.253170 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 361/696] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:46 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 1554 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:46.269000 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f482afec-e6a0-4b46-8391-fc9d17f57721 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:46.269554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f482afec-e6a0-4b46-8391-fc9d17f57721 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:46.272198 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:46.273010 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=10 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:46.279288 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8f2cebc9-3f10-4625-874a-4025434ada1f tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:46.280116 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8f2cebc9-3f10-4625-874a-4025434ada1f tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:46.280263 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f482afec-e6a0-4b46-8391-fc9d17f57721 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:46.280635 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f482afec-e6a0-4b46-8391-fc9d17f57721 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:46.281491 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f482afec-e6a0-4b46-8391-fc9d17f57721 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:46.281833 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f482afec-e6a0-4b46-8391-fc9d17f57721 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:46.286852 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8f2cebc9-3f10-4625-874a-4025434ada1f tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:46.287380 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8f2cebc9-3f10-4625-874a-4025434ada1f tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:46.288230 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8f2cebc9-3f10-4625-874a-4025434ada1f tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:46.288695 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8f2cebc9-3f10-4625-874a-4025434ada1f tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:46.389863 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f482afec-e6a0-4b46-8391-fc9d17f57721 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.124635 Jul 05 10:17:46.390585 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 336/697] 10.222.0.20 () {58 vars in 1274 bytes} [Tue Jul 5 10:17:46 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/os-interface => generated 284 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:17:46.541110 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8f2cebc9-3f10-4625-874a-4025434ada1f tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:46.549027 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8f2cebc9-3f10-4625-874a-4025434ada1f tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1509 microversion: 2.1 time: 0.273440 Jul 05 10:17:46.549601 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 362/698] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:46 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1509 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:46.565727 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-25fbf464-2875-4c7c-b147-1e7b2aa7e263 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:17:46.566289 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-25fbf464-2875-4c7c-b147-1e7b2aa7e263 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:46.572801 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-25fbf464-2875-4c7c-b147-1e7b2aa7e263 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:46.573254 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-25fbf464-2875-4c7c-b147-1e7b2aa7e263 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:46.574022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-25fbf464-2875-4c7c-b147-1e7b2aa7e263 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:46.574461 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-25fbf464-2875-4c7c-b147-1e7b2aa7e263 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:46.657640 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:46.680479 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-25fbf464-2875-4c7c-b147-1e7b2aa7e263 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "POST /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b/action" status: 202 len: 0 microversion: 2.1 time: 0.118517 Jul 05 10:17:46.681061 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 337/699] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:17:46 2022] POST /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b/action => generated 0 bytes in 120 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:17:46.689689 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-41e1ddc6-9171-40b8-8135-fb617ab9b005 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:46.690525 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-41e1ddc6-9171-40b8-8135-fb617ab9b005 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:46.698938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-41e1ddc6-9171-40b8-8135-fb617ab9b005 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:46.699590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-41e1ddc6-9171-40b8-8135-fb617ab9b005 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:46.700772 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-41e1ddc6-9171-40b8-8135-fb617ab9b005 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:46.701452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-41e1ddc6-9171-40b8-8135-fb617ab9b005 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:47.172490 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-41e1ddc6-9171-40b8-8135-fb617ab9b005 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:47.182704 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-41e1ddc6-9171-40b8-8135-fb617ab9b005 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1517 microversion: 2.1 time: 0.497119 Jul 05 10:17:47.183819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 363/700] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:46 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1517 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:47.470982 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-307fa2bd-ab82-49bf-be0b-7058760cc49a req-75a8863c-2032-45cd-a3ec-17b2c8264bdd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4684ec3d-0407-461f-8a19-d6d8f99ca49b", "name": "network-vif-unplugged", "status": "completed", "tag": "1a0fe351-4265-449d-8b7d-508a71c44b30"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:17:47.486295 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-307fa2bd-ab82-49bf-be0b-7058760cc49a req-75a8863c-2032-45cd-a3ec-17b2c8264bdd service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:47.486739 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-307fa2bd-ab82-49bf-be0b-7058760cc49a req-75a8863c-2032-45cd-a3ec-17b2c8264bdd service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:47.497885 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.compute.server_external_events [req-307fa2bd-ab82-49bf-be0b-7058760cc49a req-75a8863c-2032-45cd-a3ec-17b2c8264bdd service nova] Unable to find a host for instance 4684ec3d-0407-461f-8a19-d6d8f99ca49b. Dropping event network-vif-unplugged {{(pid=117812) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Jul 05 10:17:47.498952 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-307fa2bd-ab82-49bf-be0b-7058760cc49a req-75a8863c-2032-45cd-a3ec-17b2c8264bdd service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.033165 Jul 05 10:17:47.499895 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 338/701] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:17:47 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 35 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Jul 05 10:17:47.707621 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-3ce308f6-2bfb-4b9d-8f6d-698e08e44205 req-1e950ff6-ef53-4e26-a37a-241eff1104b3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "70cdd8c1-3058-4809-a0d1-73e39cf2e520", "tag": "dbd17c56-2b84-42e7-9867-b2fcb6dc597e"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:17:47.716619 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-3ce308f6-2bfb-4b9d-8f6d-698e08e44205 req-1e950ff6-ef53-4e26-a37a-241eff1104b3 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:47.717064 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-3ce308f6-2bfb-4b9d-8f6d-698e08e44205 req-1e950ff6-ef53-4e26-a37a-241eff1104b3 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:47.727984 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-3ce308f6-2bfb-4b9d-8f6d-698e08e44205 req-1e950ff6-ef53-4e26-a37a-241eff1104b3 service nova] Creating event network-changed:dbd17c56-2b84-42e7-9867-b2fcb6dc597e for instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 on hv-cinder-maste Jul 05 10:17:47.733205 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-3ce308f6-2bfb-4b9d-8f6d-698e08e44205 req-1e950ff6-ef53-4e26-a37a-241eff1104b3 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029221 Jul 05 10:17:47.733923 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 364/702] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:17:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:17:48.200267 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-cb09ea75-95b9-473e-a6a1-46be01b2f159 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:48.201012 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-cb09ea75-95b9-473e-a6a1-46be01b2f159 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:48.207576 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cb09ea75-95b9-473e-a6a1-46be01b2f159 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:48.208351 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cb09ea75-95b9-473e-a6a1-46be01b2f159 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:48.209397 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cb09ea75-95b9-473e-a6a1-46be01b2f159 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:48.210033 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cb09ea75-95b9-473e-a6a1-46be01b2f159 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:48.395598 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-cb09ea75-95b9-473e-a6a1-46be01b2f159 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:48.404967 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-cb09ea75-95b9-473e-a6a1-46be01b2f159 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1519 microversion: 2.1 time: 0.208180 Jul 05 10:17:48.405585 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 339/703] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:48 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1519 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:49.423975 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a95fc069-1e0c-4339-8a17-663f2408bb85 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:49.424695 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a95fc069-1e0c-4339-8a17-663f2408bb85 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:49.431421 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a95fc069-1e0c-4339-8a17-663f2408bb85 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:49.431868 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a95fc069-1e0c-4339-8a17-663f2408bb85 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:49.432647 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a95fc069-1e0c-4339-8a17-663f2408bb85 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:49.433081 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a95fc069-1e0c-4339-8a17-663f2408bb85 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:49.561645 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a95fc069-1e0c-4339-8a17-663f2408bb85 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:49.569111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a95fc069-1e0c-4339-8a17-663f2408bb85 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1575 microversion: 2.1 time: 0.149801 Jul 05 10:17:49.569701 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 365/704] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:49 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1575 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:50.585719 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6a66201b-2b02-46ed-bd2c-fca2a4b7fc7d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:50.586326 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6a66201b-2b02-46ed-bd2c-fca2a4b7fc7d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:50.592820 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6a66201b-2b02-46ed-bd2c-fca2a4b7fc7d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:50.593270 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6a66201b-2b02-46ed-bd2c-fca2a4b7fc7d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:50.594032 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6a66201b-2b02-46ed-bd2c-fca2a4b7fc7d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:50.594469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6a66201b-2b02-46ed-bd2c-fca2a4b7fc7d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:50.727860 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6a66201b-2b02-46ed-bd2c-fca2a4b7fc7d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:50.735590 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6a66201b-2b02-46ed-bd2c-fca2a4b7fc7d tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1575 microversion: 2.1 time: 0.153754 Jul 05 10:17:50.736165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 340/705] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:50 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1575 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:51.753437 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1295c6bd-7353-4631-a1b4-776ad08a39a2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:51.754017 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1295c6bd-7353-4631-a1b4-776ad08a39a2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:51.761121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1295c6bd-7353-4631-a1b4-776ad08a39a2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:51.761572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1295c6bd-7353-4631-a1b4-776ad08a39a2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:51.762324 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1295c6bd-7353-4631-a1b4-776ad08a39a2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:51.762753 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1295c6bd-7353-4631-a1b4-776ad08a39a2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:51.892676 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1295c6bd-7353-4631-a1b4-776ad08a39a2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:51.900055 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1295c6bd-7353-4631-a1b4-776ad08a39a2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1575 microversion: 2.1 time: 0.150467 Jul 05 10:17:51.900648 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 366/706] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:51 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1575 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:17:52.916662 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e35c293e-738a-4ca9-b449-c7c22153df7c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:52.917271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e35c293e-738a-4ca9-b449-c7c22153df7c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:52.923775 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e35c293e-738a-4ca9-b449-c7c22153df7c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:52.924222 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e35c293e-738a-4ca9-b449-c7c22153df7c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:52.925068 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e35c293e-738a-4ca9-b449-c7c22153df7c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:52.925429 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e35c293e-738a-4ca9-b449-c7c22153df7c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:53.059817 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e35c293e-738a-4ca9-b449-c7c22153df7c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:53.067278 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e35c293e-738a-4ca9-b449-c7c22153df7c tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1575 microversion: 2.1 time: 0.154481 Jul 05 10:17:53.067842 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 341/707] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:52 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1575 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:54.084127 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-444c7735-53b2-4a29-9dbe-c79fb042bddc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:54.085230 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-444c7735-53b2-4a29-9dbe-c79fb042bddc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:54.092048 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-444c7735-53b2-4a29-9dbe-c79fb042bddc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:54.092524 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-444c7735-53b2-4a29-9dbe-c79fb042bddc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:54.093272 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-444c7735-53b2-4a29-9dbe-c79fb042bddc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:54.093696 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-444c7735-53b2-4a29-9dbe-c79fb042bddc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:54.219937 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-444c7735-53b2-4a29-9dbe-c79fb042bddc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:54.227846 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-444c7735-53b2-4a29-9dbe-c79fb042bddc tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1575 microversion: 2.1 time: 0.147553 Jul 05 10:17:54.228452 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 367/708] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:54 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1575 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:55.245327 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-41cbfff8-637a-4d3a-b9dc-60998b836840 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:55.245840 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-41cbfff8-637a-4d3a-b9dc-60998b836840 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:55.252271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-41cbfff8-637a-4d3a-b9dc-60998b836840 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:55.252747 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-41cbfff8-637a-4d3a-b9dc-60998b836840 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:55.253498 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-41cbfff8-637a-4d3a-b9dc-60998b836840 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:55.253934 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-41cbfff8-637a-4d3a-b9dc-60998b836840 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:55.380840 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-41cbfff8-637a-4d3a-b9dc-60998b836840 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:55.388213 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-41cbfff8-637a-4d3a-b9dc-60998b836840 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1575 microversion: 2.1 time: 0.147449 Jul 05 10:17:55.388801 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 342/709] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:55 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1575 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:56.404816 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5de22c2e-a628-455d-8bd8-8ed296f13a7a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:56.405456 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5de22c2e-a628-455d-8bd8-8ed296f13a7a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:56.412013 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5de22c2e-a628-455d-8bd8-8ed296f13a7a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:56.412491 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5de22c2e-a628-455d-8bd8-8ed296f13a7a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:56.413259 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5de22c2e-a628-455d-8bd8-8ed296f13a7a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:56.413681 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5de22c2e-a628-455d-8bd8-8ed296f13a7a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:56.431564 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=100 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:17:56.545767 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5de22c2e-a628-455d-8bd8-8ed296f13a7a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:56.553680 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5de22c2e-a628-455d-8bd8-8ed296f13a7a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1575 microversion: 2.1 time: 0.152222 Jul 05 10:17:56.554239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 368/710] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:56 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1575 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:57.570316 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a5e60060-c95c-481d-a206-be768ae6cdf2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:57.571935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a5e60060-c95c-481d-a206-be768ae6cdf2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:57.579800 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5e60060-c95c-481d-a206-be768ae6cdf2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:57.580241 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5e60060-c95c-481d-a206-be768ae6cdf2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:57.581034 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5e60060-c95c-481d-a206-be768ae6cdf2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:57.581468 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5e60060-c95c-481d-a206-be768ae6cdf2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:57.712308 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a5e60060-c95c-481d-a206-be768ae6cdf2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:57.719707 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a5e60060-c95c-481d-a206-be768ae6cdf2 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1575 microversion: 2.1 time: 0.154051 Jul 05 10:17:57.720271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 343/711] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:57 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1575 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:58.736287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-85331376-bf95-45f7-808b-e2296c06adeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:58.736920 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-85331376-bf95-45f7-808b-e2296c06adeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:58.742921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-85331376-bf95-45f7-808b-e2296c06adeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:58.743363 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-85331376-bf95-45f7-808b-e2296c06adeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:58.744116 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-85331376-bf95-45f7-808b-e2296c06adeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:58.744586 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-85331376-bf95-45f7-808b-e2296c06adeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:58.876650 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-85331376-bf95-45f7-808b-e2296c06adeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:17:58.884685 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-85331376-bf95-45f7-808b-e2296c06adeb tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1575 microversion: 2.1 time: 0.152285 Jul 05 10:17:58.885247 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 369/712] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:58 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1575 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:17:59.559385 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-f9b25baf-5ca1-48dd-96ec-44be0ad88d64 req-4b2ea40c-105b-443e-8743-77023127e67f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4684ec3d-0407-461f-8a19-d6d8f99ca49b", "name": "network-vif-plugged", "status": "completed", "tag": "1a0fe351-4265-449d-8b7d-508a71c44b30"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:17:59.568303 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-f9b25baf-5ca1-48dd-96ec-44be0ad88d64 req-4b2ea40c-105b-443e-8743-77023127e67f service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:59.568776 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-f9b25baf-5ca1-48dd-96ec-44be0ad88d64 req-4b2ea40c-105b-443e-8743-77023127e67f service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:59.579997 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-f9b25baf-5ca1-48dd-96ec-44be0ad88d64 req-4b2ea40c-105b-443e-8743-77023127e67f service nova] Creating event network-vif-plugged:1a0fe351-4265-449d-8b7d-508a71c44b30 for instance 4684ec3d-0407-461f-8a19-d6d8f99ca49b on hv-cinder-maste Jul 05 10:17:59.585053 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-f9b25baf-5ca1-48dd-96ec-44be0ad88d64 req-4b2ea40c-105b-443e-8743-77023127e67f service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029316 Jul 05 10:17:59.585813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 344/713] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:17:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:17:59.787432 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b711f334-7f4e-4072-a2d6-a0160a2d6f99 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:59.787953 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b711f334-7f4e-4072-a2d6-a0160a2d6f99 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:59.794871 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b711f334-7f4e-4072-a2d6-a0160a2d6f99 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:59.795315 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b711f334-7f4e-4072-a2d6-a0160a2d6f99 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:59.796066 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b711f334-7f4e-4072-a2d6-a0160a2d6f99 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:59.796528 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b711f334-7f4e-4072-a2d6-a0160a2d6f99 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:59.878407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.objects.instance [None req-b711f334-7f4e-4072-a2d6-a0160a2d6f99 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lazy-loading 'services' on Instance uuid 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd {{(pid=117813) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:17:59.901594 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-375bf64b-1041-4ad1-909b-3f504f87a091 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:59.902328 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-375bf64b-1041-4ad1-909b-3f504f87a091 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:59.908724 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-375bf64b-1041-4ad1-909b-3f504f87a091 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:59.909183 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-375bf64b-1041-4ad1-909b-3f504f87a091 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:59.910031 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-375bf64b-1041-4ad1-909b-3f504f87a091 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:59.910659 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-375bf64b-1041-4ad1-909b-3f504f87a091 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:59.962128 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b711f334-7f4e-4072-a2d6-a0160a2d6f99 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "DELETE /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments/a2c26058-b0da-463f-bc49-c3777e966958" status: 202 len: 0 microversion: 2.1 time: 0.178331 Jul 05 10:17:59.962758 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 370/714] 10.222.0.20 () {58 vars in 1415 bytes} [Tue Jul 5 10:17:59 2022] DELETE /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments/a2c26058-b0da-463f-bc49-c3777e966958 => generated 0 bytes in 180 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:17:59.971158 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3618535e-6492-470d-93d5-c03b3e68a64c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:17:59.972172 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3618535e-6492-470d-93d5-c03b3e68a64c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:17:59.978974 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3618535e-6492-470d-93d5-c03b3e68a64c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:59.979471 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3618535e-6492-470d-93d5-c03b3e68a64c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:17:59.980568 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3618535e-6492-470d-93d5-c03b3e68a64c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:17:59.980949 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3618535e-6492-470d-93d5-c03b3e68a64c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:00.009575 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3618535e-6492-470d-93d5-c03b3e68a64c tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042403 Jul 05 10:18:00.010349 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 371/715] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:17:59 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:18:00.051107 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-375bf64b-1041-4ad1-909b-3f504f87a091 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:00.058677 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-375bf64b-1041-4ad1-909b-3f504f87a091 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1575 microversion: 2.1 time: 0.161168 Jul 05 10:18:00.059241 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 345/716] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:17:59 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1575 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:01.021063 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9a33e467-fac0-44d0-9f1f-c3f3e89d86a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:01.021981 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9a33e467-fac0-44d0-9f1f-c3f3e89d86a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:01.028292 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9a33e467-fac0-44d0-9f1f-c3f3e89d86a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:01.028784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9a33e467-fac0-44d0-9f1f-c3f3e89d86a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:01.029535 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9a33e467-fac0-44d0-9f1f-c3f3e89d86a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:01.029965 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9a33e467-fac0-44d0-9f1f-c3f3e89d86a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:01.058619 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9a33e467-fac0-44d0-9f1f-c3f3e89d86a1 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041458 Jul 05 10:18:01.059162 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 372/717] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:18:01 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:18:01.074795 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-398310ef-7773-4c69-ab1c-c001d62dafc7 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:01.075405 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-398310ef-7773-4c69-ab1c-c001d62dafc7 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:01.081560 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-398310ef-7773-4c69-ab1c-c001d62dafc7 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:01.082012 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-398310ef-7773-4c69-ab1c-c001d62dafc7 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:01.082770 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-398310ef-7773-4c69-ab1c-c001d62dafc7 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:01.083206 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-398310ef-7773-4c69-ab1c-c001d62dafc7 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:01.217379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-398310ef-7773-4c69-ab1c-c001d62dafc7 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:01.225112 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-398310ef-7773-4c69-ab1c-c001d62dafc7 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1575 microversion: 2.1 time: 0.154110 Jul 05 10:18:01.226163 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 346/718] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:01 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1575 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:02.072735 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2ca6ec36-c1da-4853-9cb7-e6b9c11f516a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:02.073646 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2ca6ec36-c1da-4853-9cb7-e6b9c11f516a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:02.080640 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2ca6ec36-c1da-4853-9cb7-e6b9c11f516a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:02.081084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2ca6ec36-c1da-4853-9cb7-e6b9c11f516a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:02.081843 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2ca6ec36-c1da-4853-9cb7-e6b9c11f516a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:02.082274 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2ca6ec36-c1da-4853-9cb7-e6b9c11f516a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:02.113695 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2ca6ec36-c1da-4853-9cb7-e6b9c11f516a tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046931 Jul 05 10:18:02.114326 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 373/719] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:18:02 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:18:02.242678 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e8f1441f-f37b-4826-94a5-975428c9b03a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:02.243347 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e8f1441f-f37b-4826-94a5-975428c9b03a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:02.249830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e8f1441f-f37b-4826-94a5-975428c9b03a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:02.250282 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e8f1441f-f37b-4826-94a5-975428c9b03a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:02.251052 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e8f1441f-f37b-4826-94a5-975428c9b03a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:02.251489 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e8f1441f-f37b-4826-94a5-975428c9b03a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:02.381106 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e8f1441f-f37b-4826-94a5-975428c9b03a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:02.389105 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e8f1441f-f37b-4826-94a5-975428c9b03a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1562 microversion: 2.1 time: 0.150871 Jul 05 10:18:02.389697 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 347/720] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:02 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1562 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:03.125481 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d6907500-4dcb-4f59-a44c-aed01b90367e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:03.126407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d6907500-4dcb-4f59-a44c-aed01b90367e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:03.132806 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d6907500-4dcb-4f59-a44c-aed01b90367e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:03.133256 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d6907500-4dcb-4f59-a44c-aed01b90367e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:03.134016 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d6907500-4dcb-4f59-a44c-aed01b90367e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:03.134451 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d6907500-4dcb-4f59-a44c-aed01b90367e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:03.162332 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d6907500-4dcb-4f59-a44c-aed01b90367e tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.040977 Jul 05 10:18:03.162883 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 374/721] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:18:03 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd/os-volume_attachments => generated 25 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 05 10:18:03.236072 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-40c47759-eda3-4bf0-bcbb-372a43455002 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:03.236611 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-40c47759-eda3-4bf0-bcbb-372a43455002 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:03.243326 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-40c47759-eda3-4bf0-bcbb-372a43455002 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:03.243893 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-40c47759-eda3-4bf0-bcbb-372a43455002 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:03.244538 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-40c47759-eda3-4bf0-bcbb-372a43455002 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:03.244972 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-40c47759-eda3-4bf0-bcbb-372a43455002 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:03.271504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-40c47759-eda3-4bf0-bcbb-372a43455002 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Going to try to terminate instance {{(pid=117812) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:18:03.344442 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-40c47759-eda3-4bf0-bcbb-372a43455002 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "DELETE /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 204 len: 0 microversion: 2.1 time: 0.111797 Jul 05 10:18:03.345319 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 348/722] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:18:03 2022] DELETE /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:18:03.353988 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ce7d6bb2-21c2-4f7e-8144-b3f176c4e0bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:03.354595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ce7d6bb2-21c2-4f7e-8144-b3f176c4e0bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:03.360891 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ce7d6bb2-21c2-4f7e-8144-b3f176c4e0bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:03.361336 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ce7d6bb2-21c2-4f7e-8144-b3f176c4e0bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:03.362095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ce7d6bb2-21c2-4f7e-8144-b3f176c4e0bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:03.362525 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ce7d6bb2-21c2-4f7e-8144-b3f176c4e0bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:03.495364 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ce7d6bb2-21c2-4f7e-8144-b3f176c4e0bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:03.502820 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ce7d6bb2-21c2-4f7e-8144-b3f176c4e0bb tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1689 microversion: 2.1 time: 0.153163 Jul 05 10:18:03.503385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 375/723] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:03 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1689 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:18:04.520142 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-210c6bcf-0cbe-41cd-9a16-821ed6d35bf9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:04.520851 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-210c6bcf-0cbe-41cd-9a16-821ed6d35bf9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:04.527188 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-210c6bcf-0cbe-41cd-9a16-821ed6d35bf9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:04.527634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-210c6bcf-0cbe-41cd-9a16-821ed6d35bf9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:04.528404 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-210c6bcf-0cbe-41cd-9a16-821ed6d35bf9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:04.528846 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-210c6bcf-0cbe-41cd-9a16-821ed6d35bf9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:04.656001 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-210c6bcf-0cbe-41cd-9a16-821ed6d35bf9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:04.663409 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-210c6bcf-0cbe-41cd-9a16-821ed6d35bf9 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1689 microversion: 2.1 time: 0.147460 Jul 05 10:18:04.663977 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 349/724] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:04 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1689 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:05.679325 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-79c9a56e-d3d7-4af5-b0ef-7900ddd4b3e8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:05.679925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-79c9a56e-d3d7-4af5-b0ef-7900ddd4b3e8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:05.686538 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-79c9a56e-d3d7-4af5-b0ef-7900ddd4b3e8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:05.686985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-79c9a56e-d3d7-4af5-b0ef-7900ddd4b3e8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:05.687741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-79c9a56e-d3d7-4af5-b0ef-7900ddd4b3e8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:05.688172 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-79c9a56e-d3d7-4af5-b0ef-7900ddd4b3e8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:05.817403 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-79c9a56e-d3d7-4af5-b0ef-7900ddd4b3e8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:05.825304 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-79c9a56e-d3d7-4af5-b0ef-7900ddd4b3e8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1689 microversion: 2.1 time: 0.149803 Jul 05 10:18:05.825866 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 376/725] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:05 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1689 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:06.842333 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-59bfcbef-173d-47bc-b180-8a7832b72494 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:06.842985 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-59bfcbef-173d-47bc-b180-8a7832b72494 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:06.849515 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59bfcbef-173d-47bc-b180-8a7832b72494 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:06.849964 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59bfcbef-173d-47bc-b180-8a7832b72494 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:06.850723 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59bfcbef-173d-47bc-b180-8a7832b72494 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:06.851165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59bfcbef-173d-47bc-b180-8a7832b72494 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:06.979733 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-59bfcbef-173d-47bc-b180-8a7832b72494 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:06.987557 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-59bfcbef-173d-47bc-b180-8a7832b72494 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1689 microversion: 2.1 time: 0.149115 Jul 05 10:18:06.988129 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 350/726] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:06 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1689 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:08.003870 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3625594d-fa92-426f-8a60-7b15907424f2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:08.004504 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3625594d-fa92-426f-8a60-7b15907424f2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:08.010983 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3625594d-fa92-426f-8a60-7b15907424f2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:08.011433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3625594d-fa92-426f-8a60-7b15907424f2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:08.012193 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3625594d-fa92-426f-8a60-7b15907424f2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:08.012654 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3625594d-fa92-426f-8a60-7b15907424f2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:08.083213 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-3625594d-fa92-426f-8a60-7b15907424f2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:08.090659 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3625594d-fa92-426f-8a60-7b15907424f2 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 200 len: 1622 microversion: 2.1 time: 0.090695 Jul 05 10:18:08.091285 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 377/727] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:07 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 1622 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:08.231966 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-a88f3547-6ff5-4d32-a0fc-584c6b6f69c7 req-ef443d58-fda3-40cf-947c-70ce886fa147 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "12e7fd00-8f3e-4fc4-882b-6ec193de0bfd", "name": "network-vif-deleted", "tag": "e27d9ee6-1685-4aec-bb2d-ae3b3206c610"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:18:08.243241 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-a88f3547-6ff5-4d32-a0fc-584c6b6f69c7 req-ef443d58-fda3-40cf-947c-70ce886fa147 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:08.244072 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-a88f3547-6ff5-4d32-a0fc-584c6b6f69c7 req-ef443d58-fda3-40cf-947c-70ce886fa147 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:08.257233 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-a88f3547-6ff5-4d32-a0fc-584c6b6f69c7 req-ef443d58-fda3-40cf-947c-70ce886fa147 service nova] Creating event network-vif-deleted:e27d9ee6-1685-4aec-bb2d-ae3b3206c610 for instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd on hv-cinder-maste Jul 05 10:18:08.263043 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-a88f3547-6ff5-4d32-a0fc-584c6b6f69c7 req-ef443d58-fda3-40cf-947c-70ce886fa147 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035946 Jul 05 10:18:08.263683 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 351/728] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:18:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:18:09.107892 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e1663968-1be5-44ca-a85b-36f8cd99d022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:09.108544 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e1663968-1be5-44ca-a85b-36f8cd99d022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:09.115334 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e1663968-1be5-44ca-a85b-36f8cd99d022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:09.115778 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e1663968-1be5-44ca-a85b-36f8cd99d022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:09.116578 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e1663968-1be5-44ca-a85b-36f8cd99d022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:09.117024 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e1663968-1be5-44ca-a85b-36f8cd99d022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:09.131035 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-e1663968-1be5-44ca-a85b-36f8cd99d022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] HTTP exception thrown: Instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd could not be found. Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e1663968-1be5-44ca-a85b-36f8cd99d022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Request method failure captured: Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd HTTP/1.1 Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4LMwEWQwEVIGHW57tb-mCZDddObx4iwQyZ5gk_llO51wkTkuVjOLNqddatV2FzjtkzWxyWr7Rmd5kPAnAKyMQJw43XAdu8pkSZT9OsjCAGydI1FCbBCPFs_pXG35EW9p3Z8AlqJqZdFJIfQwAG269sW3HEsdORJuVOabGGWKLHWOI Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 3f2199fa0bc54901aa24195a250a9e25 Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:18:09.132781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:18:09.134657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/3f2199fa0bc54901aa24195a250a9e25"}]}] Jul 05 10:18:09.134657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:18:09.134657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 3f2199fa0bc54901aa24195a250a9e25 Jul 05 10:18:09.134657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:18:09.134657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-ServerRescueNegativeTestJSON-1125341258-project Jul 05 10:18:09.134657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:18:09.134657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:18:09.134657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 248946c73cb245f9b8e6fe73e46133cf Jul 05 10:18:09.134657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1125341258-project Jul 05 10:18:09.134657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:18:09.134657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd could not be found. Jul 05 10:18:09.134657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': '12e7fd00-8f3e-4fc4-882b-6ec193de0bfd'} Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd could not be found. Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:18:09.136249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd could not be found. Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd could not be found. Jul 05 10:18:09.137625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:09.138746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e1663968-1be5-44ca-a85b-36f8cd99d022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Returning 404 to user: Instance 12e7fd00-8f3e-4fc4-882b-6ec193de0bfd could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:18:09.138746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e1663968-1be5-44ca-a85b-36f8cd99d022 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd" status: 404 len: 111 microversion: 2.1 time: 0.030977 Jul 05 10:18:09.138746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 378/729] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:09 2022] GET /compute/v2.1/servers/12e7fd00-8f3e-4fc4-882b-6ec193de0bfd => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:18:09.144285 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-85d3cae5-e9d2-4721-a53e-4fb92e593980 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '.version_select at 0x7feb895a21f0>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:09.161647 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-85d3cae5-e9d2-4721-a53e-4fb92e593980 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-537711298" status: 202 len: 0 microversion: 2.1 time: 0.021487 Jul 05 10:18:09.162215 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 352/730] 10.222.0.20 () {58 vars in 1217 bytes} [Tue Jul 5 10:18:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-537711298 => generated 0 bytes in 23 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:18:12.019540 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-dcf0b264-3e6f-4ea0-b825-c9c1f293d034 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:12.020007 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-dcf0b264-3e6f-4ea0-b825-c9c1f293d034 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:12.026917 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dcf0b264-3e6f-4ea0-b825-c9c1f293d034 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:12.027383 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dcf0b264-3e6f-4ea0-b825-c9c1f293d034 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:12.028127 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dcf0b264-3e6f-4ea0-b825-c9c1f293d034 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:12.028596 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dcf0b264-3e6f-4ea0-b825-c9c1f293d034 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:12.053310 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-dcf0b264-3e6f-4ea0-b825-c9c1f293d034 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Going to try to terminate instance {{(pid=117813) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:18:12.125768 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-dcf0b264-3e6f-4ea0-b825-c9c1f293d034 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "DELETE /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 204 len: 0 microversion: 2.1 time: 0.109590 Jul 05 10:18:12.127276 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 379/731] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:18:12 2022] DELETE /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:18:12.135276 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-36c4f69f-9e7f-4906-8729-800f1e833f20 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:12.135990 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-36c4f69f-9e7f-4906-8729-800f1e833f20 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:12.142077 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36c4f69f-9e7f-4906-8729-800f1e833f20 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:12.142529 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36c4f69f-9e7f-4906-8729-800f1e833f20 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:12.143299 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36c4f69f-9e7f-4906-8729-800f1e833f20 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:12.143736 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-36c4f69f-9e7f-4906-8729-800f1e833f20 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:12.274303 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-36c4f69f-9e7f-4906-8729-800f1e833f20 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:12.282429 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-36c4f69f-9e7f-4906-8729-800f1e833f20 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1508 microversion: 2.1 time: 0.151689 Jul 05 10:18:12.283001 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 353/732] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:12 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1508 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:13.299595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9a6eedc8-a1c5-45a4-9e1d-d7d9969a1a75 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:13.300311 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9a6eedc8-a1c5-45a4-9e1d-d7d9969a1a75 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:13.302667 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:13.308745 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9a6eedc8-a1c5-45a4-9e1d-d7d9969a1a75 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:13.309196 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9a6eedc8-a1c5-45a4-9e1d-d7d9969a1a75 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:13.309958 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9a6eedc8-a1c5-45a4-9e1d-d7d9969a1a75 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:13.310387 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9a6eedc8-a1c5-45a4-9e1d-d7d9969a1a75 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:13.518476 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9a6eedc8-a1c5-45a4-9e1d-d7d9969a1a75 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:13.526382 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9a6eedc8-a1c5-45a4-9e1d-d7d9969a1a75 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 200 len: 1508 microversion: 2.1 time: 0.231086 Jul 05 10:18:13.526946 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 380/733] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:13 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 1508 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:13.917724 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-f2df7513-6af1-48d6-83f9-ffecccbecd66 req-45aa4298-9702-4d74-8ff6-a17bc81ca8ec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fc065612-7f7d-45fb-89cc-9f332fb15dd1", "name": "network-vif-deleted", "tag": "ff58dc45-3b6a-4ad9-a225-3ec518174c19"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:18:13.922905 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=52,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:13.930389 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-f2df7513-6af1-48d6-83f9-ffecccbecd66 req-45aa4298-9702-4d74-8ff6-a17bc81ca8ec service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:13.931642 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-f2df7513-6af1-48d6-83f9-ffecccbecd66 req-45aa4298-9702-4d74-8ff6-a17bc81ca8ec service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:13.944426 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-f2df7513-6af1-48d6-83f9-ffecccbecd66 req-45aa4298-9702-4d74-8ff6-a17bc81ca8ec service nova] Creating event network-vif-deleted:ff58dc45-3b6a-4ad9-a225-3ec518174c19 for instance fc065612-7f7d-45fb-89cc-9f332fb15dd1 on hv-cinder-maste Jul 05 10:18:13.950157 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-f2df7513-6af1-48d6-83f9-ffecccbecd66 req-45aa4298-9702-4d74-8ff6-a17bc81ca8ec service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036351 Jul 05 10:18:13.951034 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 354/734] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:18:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:18:14.543560 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f918d63b-fcfc-47f6-970b-c02aef188bb7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:14.544095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f918d63b-fcfc-47f6-970b-c02aef188bb7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: fc065612-7f7d-45fb-89cc-9f332fb15dd1] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:14.553071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f918d63b-fcfc-47f6-970b-c02aef188bb7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:14.553518 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f918d63b-fcfc-47f6-970b-c02aef188bb7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:14.554810 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f918d63b-fcfc-47f6-970b-c02aef188bb7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:14.555247 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f918d63b-fcfc-47f6-970b-c02aef188bb7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:14.569005 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-f918d63b-fcfc-47f6-970b-c02aef188bb7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] HTTP exception thrown: Instance fc065612-7f7d-45fb-89cc-9f332fb15dd1 could not be found. Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f918d63b-fcfc-47f6-970b-c02aef188bb7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Request method failure captured: Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 HTTP/1.1 Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4LMwEWQwEVIGHW57tb-mCZDddObx4iwQyZ5gk_llO51wkTkuVjOLNqddatV2FzjtkzWxyWr7Rmd5kPAnAKyMQJw43XAdu8pkSZT9OsjCAGydI1FCbBCPFs_pXG35EW9p3Z8AlqJqZdFJIfQwAG269sW3HEsdORJuVOabGGWKLHWOI Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 3f2199fa0bc54901aa24195a250a9e25 Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:18:14.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:18:14.572117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/3f2199fa0bc54901aa24195a250a9e25"}]}] Jul 05 10:18:14.572117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:18:14.572117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 3f2199fa0bc54901aa24195a250a9e25 Jul 05 10:18:14.572117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:18:14.572117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-ServerRescueNegativeTestJSON-1125341258-project Jul 05 10:18:14.572117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:18:14.572117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:18:14.572117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 248946c73cb245f9b8e6fe73e46133cf Jul 05 10:18:14.572117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1125341258-project Jul 05 10:18:14.572117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:18:14.572117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance fc065612-7f7d-45fb-89cc-9f332fb15dd1 could not be found. Jul 05 10:18:14.572117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'fc065612-7f7d-45fb-89cc-9f332fb15dd1'} Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fc065612-7f7d-45fb-89cc-9f332fb15dd1 could not be found. Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:18:14.572963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fc065612-7f7d-45fb-89cc-9f332fb15dd1 could not be found. Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fc065612-7f7d-45fb-89cc-9f332fb15dd1 could not be found. Jul 05 10:18:14.574138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:14.575239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f918d63b-fcfc-47f6-970b-c02aef188bb7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Returning 404 to user: Instance fc065612-7f7d-45fb-89cc-9f332fb15dd1 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:18:14.575239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f918d63b-fcfc-47f6-970b-c02aef188bb7 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1" status: 404 len: 111 microversion: 2.1 time: 0.032870 Jul 05 10:18:14.575239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 381/735] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:14 2022] GET /compute/v2.1/servers/fc065612-7f7d-45fb-89cc-9f332fb15dd1 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:18:14.581059 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-cbc7f629-e0b4-46a6-b416-81f3d561446d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:14.581642 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-cbc7f629-e0b4-46a6-b416-81f3d561446d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:14.587762 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cbc7f629-e0b4-46a6-b416-81f3d561446d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:14.588221 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cbc7f629-e0b4-46a6-b416-81f3d561446d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:14.589028 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cbc7f629-e0b4-46a6-b416-81f3d561446d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:14.589477 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cbc7f629-e0b4-46a6-b416-81f3d561446d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:14.614272 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-cbc7f629-e0b4-46a6-b416-81f3d561446d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Going to try to terminate instance {{(pid=117812) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:18:14.628280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:14.687682 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-cbc7f629-e0b4-46a6-b416-81f3d561446d tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "DELETE /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 204 len: 0 microversion: 2.1 time: 0.110523 Jul 05 10:18:14.688192 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 355/736] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:18:14 2022] DELETE /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:18:14.698146 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-682f7bf8-3d4f-473f-ab2e-3b6b6c15c7d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:14.698747 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-682f7bf8-3d4f-473f-ab2e-3b6b6c15c7d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:14.704971 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-682f7bf8-3d4f-473f-ab2e-3b6b6c15c7d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:14.705412 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-682f7bf8-3d4f-473f-ab2e-3b6b6c15c7d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:14.706161 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-682f7bf8-3d4f-473f-ab2e-3b6b6c15c7d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:14.706590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-682f7bf8-3d4f-473f-ab2e-3b6b6c15c7d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:14.842814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-682f7bf8-3d4f-473f-ab2e-3b6b6c15c7d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:14.850410 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-682f7bf8-3d4f-473f-ab2e-3b6b6c15c7d8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.157887 Jul 05 10:18:14.850966 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 382/737] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:14 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:15.869002 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-1e8c9299-87e9-4efa-978c-72b9b654af17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:15.869729 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-1e8c9299-87e9-4efa-978c-72b9b654af17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:15.876192 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1e8c9299-87e9-4efa-978c-72b9b654af17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:15.876681 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1e8c9299-87e9-4efa-978c-72b9b654af17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:15.877450 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1e8c9299-87e9-4efa-978c-72b9b654af17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:15.877889 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1e8c9299-87e9-4efa-978c-72b9b654af17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:16.098373 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1e8c9299-87e9-4efa-978c-72b9b654af17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '248946c73cb245f9b8e6fe73e46133cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f2199fa0bc54901aa24195a250a9e25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:16.106995 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-1e8c9299-87e9-4efa-978c-72b9b654af17 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 200 len: 1521 microversion: 2.1 time: 0.242399 Jul 05 10:18:16.116883 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 356/738] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:15 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 1521 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:16.339201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-7011c7d8-042d-4e76-8abc-75de4c517c80 req-eab68511-853c-4ad5-855e-d109df1fa890 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f6c29e5f-2d52-4c7a-9122-11dbaf998421", "name": "network-vif-deleted", "tag": "494e9e9b-34e3-4ca8-b312-38f821ddd851"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:18:16.347823 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-7011c7d8-042d-4e76-8abc-75de4c517c80 req-eab68511-853c-4ad5-855e-d109df1fa890 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:16.348253 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-7011c7d8-042d-4e76-8abc-75de4c517c80 req-eab68511-853c-4ad5-855e-d109df1fa890 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:16.359595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-7011c7d8-042d-4e76-8abc-75de4c517c80 req-eab68511-853c-4ad5-855e-d109df1fa890 service nova] Creating event network-vif-deleted:494e9e9b-34e3-4ca8-b312-38f821ddd851 for instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 on hv-cinder-maste Jul 05 10:18:16.364440 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-7011c7d8-042d-4e76-8abc-75de4c517c80 req-eab68511-853c-4ad5-855e-d109df1fa890 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028861 Jul 05 10:18:16.365027 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 383/739] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:18:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:18:17.124430 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b632bd6a-3fa7-495f-9e0f-00dd587aaed8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:17.125041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b632bd6a-3fa7-495f-9e0f-00dd587aaed8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] [instance: f6c29e5f-2d52-4c7a-9122-11dbaf998421] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:17.131391 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b632bd6a-3fa7-495f-9e0f-00dd587aaed8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:17.131835 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b632bd6a-3fa7-495f-9e0f-00dd587aaed8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:17.132620 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b632bd6a-3fa7-495f-9e0f-00dd587aaed8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:17.133053 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b632bd6a-3fa7-495f-9e0f-00dd587aaed8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:17.146454 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-b632bd6a-3fa7-495f-9e0f-00dd587aaed8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] HTTP exception thrown: Instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 could not be found. Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b632bd6a-3fa7-495f-9e0f-00dd587aaed8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Request method failure captured: Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 HTTP/1.1 Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA4LMwEWQwEVIGHW57tb-mCZDddObx4iwQyZ5gk_llO51wkTkuVjOLNqddatV2FzjtkzWxyWr7Rmd5kPAnAKyMQJw43XAdu8pkSZT9OsjCAGydI1FCbBCPFs_pXG35EW9p3Z8AlqJqZdFJIfQwAG269sW3HEsdORJuVOabGGWKLHWOI Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: 3f2199fa0bc54901aa24195a250a9e25 Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:18:17.148041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:18:17.149597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/3f2199fa0bc54901aa24195a250a9e25"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/3f2199fa0bc54901aa24195a250a9e25"}]}] Jul 05 10:18:17.149597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:18:17.149597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: 3f2199fa0bc54901aa24195a250a9e25 Jul 05 10:18:17.149597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1125341258 Jul 05 10:18:17.149597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-ServerRescueNegativeTestJSON-1125341258-project Jul 05 10:18:17.149597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:18:17.149597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:18:17.149597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: 248946c73cb245f9b8e6fe73e46133cf Jul 05 10:18:17.149597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1125341258-project Jul 05 10:18:17.149597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:18:17.149597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 could not be found. Jul 05 10:18:17.149597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': 'f6c29e5f-2d52-4c7a-9122-11dbaf998421'} Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 could not be found. Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:18:17.150421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 could not be found. Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 could not be found. Jul 05 10:18:17.151627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:18:17.152702 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b632bd6a-3fa7-495f-9e0f-00dd587aaed8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] Returning 404 to user: Instance f6c29e5f-2d52-4c7a-9122-11dbaf998421 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:18:17.152702 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b632bd6a-3fa7-495f-9e0f-00dd587aaed8 tempest-ServerRescueNegativeTestJSON-1125341258 tempest-ServerRescueNegativeTestJSON-1125341258-project] 10.222.0.20 "GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421" status: 404 len: 111 microversion: 2.1 time: 0.028869 Jul 05 10:18:17.152702 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 357/740] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:17 2022] GET /compute/v2.1/servers/f6c29e5f-2d52-4c7a-9122-11dbaf998421 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:18:25.658651 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8e0a6866-b855-4cb0-b563-7c5cd0d2de06 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Action: 'create', calling method: .version_select at 0x7feb8df70550>, body: {"keypair": {"name": "tempest-keypair-691061805", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMSfghL2xNYvhqxEIgBQh8YykBl7YtUQ1FQMf4HVtfMc5Yew7VxzwvthHqnLb0X5kxR9flVxIdatdt4MLpmBcQA2l4uaEfbS2XUZ5WKaKYYy58VeYU3J1uef7/MpYdmXXQ=="}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:18:25.677449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-8e0a6866-b855-4cb0-b563-7c5cd0d2de06 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Getting quotas for project f0c19d009f1b4ef09a40267bcf654107. Resources: {'key_pairs'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:18:25.682720 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-8e0a6866-b855-4cb0-b563-7c5cd0d2de06 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Getting quotas for user d4a608563189457dbcadf8d48acdd84a and project f0c19d009f1b4ef09a40267bcf654107. Resources: {'key_pairs'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:18:25.725890 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8e0a6866-b855-4cb0-b563-7c5cd0d2de06 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.186960 Jul 05 10:18:25.726776 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 384/741] 10.222.0.20 () {62 vars in 1199 bytes} [Tue Jul 5 10:18:25 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 189 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 05 10:18:27.045510 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1324988284", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "fddec1cf-8b7c-4e90-af09-12a66276106e"}], "security_groups": [{"name": "tempest-securitygroup--1727976110"}], "key_name": "tempest-keypair-691061805", "user_data": "}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:18:27.139825 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=14,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:27.157936 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.network.neutron [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] validate_networks() for [('fddec1cf-8b7c-4e90-af09-12a66276106e', None, None, None, None, None)] {{(pid=117812) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:18:27.543643 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Flavor limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:18:27.544089 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Image limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:18:27.544612 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Flavor pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:18:27.545015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Image pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:18:27.545418 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:18:27.554029 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:27.554475 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:27.555258 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:27.555692 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:27.571891 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Getting quotas for project f0c19d009f1b4ef09a40267bcf654107. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:18:27.576298 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Getting quotas for user d4a608563189457dbcadf8d48acdd84a and project f0c19d009f1b4ef09a40267bcf654107. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:18:27.587008 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Going to run 1 instances... {{(pid=117812) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:18:27.589949 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117812) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:18:27.638060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-bdc202cb-1d9b-4ecc-b6ef-9f23408e45d1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.597754 Jul 05 10:18:27.638879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 358/742] 10.222.0.20 () {62 vars in 1187 bytes} [Tue Jul 5 10:18:27 2022] POST /compute/v2.1/servers => generated 402 bytes in 599 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 05 10:18:27.648685 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-51684c03-0bec-4e0f-9e14-1af0debecc60 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:27.649678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-51684c03-0bec-4e0f-9e14-1af0debecc60 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:27.653213 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:27.724195 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-51684c03-0bec-4e0f-9e14-1af0debecc60 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:27.733186 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-51684c03-0bec-4e0f-9e14-1af0debecc60 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:27.734943 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-51684c03-0bec-4e0f-9e14-1af0debecc60 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:27.736067 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-51684c03-0bec-4e0f-9e14-1af0debecc60 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1281 microversion: 2.20 time: 0.091468 Jul 05 10:18:27.737019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 385/743] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:27 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1281 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:18:28.523958 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0c7cb6c7-4c17-45e3-9b6e-0ee6a54cad06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-179336894"}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:18:28.524512 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0c7cb6c7-4c17-45e3-9b6e-0ee6a54cad06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:28.534847 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0c7cb6c7-4c17-45e3-9b6e-0ee6a54cad06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:28.538329 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0c7cb6c7-4c17-45e3-9b6e-0ee6a54cad06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:28.540868 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0c7cb6c7-4c17-45e3-9b6e-0ee6a54cad06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:28.541325 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0c7cb6c7-4c17-45e3-9b6e-0ee6a54cad06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:28.716384 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.network.security_group_api [None req-0c7cb6c7-4c17-45e3-9b6e-0ee6a54cad06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Removing security group ec33f5ae-fb77-4249-a272-9317a9186cf4 from port 080193f7-c96f-450e-87cc-cab454029efe Jul 05 10:18:28.755449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-56061451-e9a6-4338-a811-291d8ef96300 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:28.756516 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-56061451-e9a6-4338-a811-291d8ef96300 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:28.763974 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-56061451-e9a6-4338-a811-291d8ef96300 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:28.764796 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-56061451-e9a6-4338-a811-291d8ef96300 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:28.765690 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-56061451-e9a6-4338-a811-291d8ef96300 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:28.766234 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-56061451-e9a6-4338-a811-291d8ef96300 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:28.852111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-56061451-e9a6-4338-a811-291d8ef96300 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:28.861062 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-56061451-e9a6-4338-a811-291d8ef96300 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:28.862819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-56061451-e9a6-4338-a811-291d8ef96300 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:28.864213 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-56061451-e9a6-4338-a811-291d8ef96300 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1333 microversion: 2.20 time: 0.112909 Jul 05 10:18:28.864944 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 386/744] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:28 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1333 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:18:29.262633 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0c7cb6c7-4c17-45e3-9b6e-0ee6a54cad06 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "POST /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/action" status: 202 len: 0 microversion: 2.1 time: 0.739484 Jul 05 10:18:29.268574 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 359/745] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:18:28 2022] POST /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/action => generated 0 bytes in 748 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:18:29.297881 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-64167304-1445-4625-9b65-1c26cf0fcff8 req-45826a47-e91d-4935-a3b5-393327601a6d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5377ac3e-4f33-4b05-bc85-80b0d7b653c4", "tag": "080193f7-c96f-450e-87cc-cab454029efe"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:18:29.316656 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-64167304-1445-4625-9b65-1c26cf0fcff8 req-45826a47-e91d-4935-a3b5-393327601a6d service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:29.317105 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-64167304-1445-4625-9b65-1c26cf0fcff8 req-45826a47-e91d-4935-a3b5-393327601a6d service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:29.336785 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-64167304-1445-4625-9b65-1c26cf0fcff8 req-45826a47-e91d-4935-a3b5-393327601a6d service nova] Creating event network-changed:080193f7-c96f-450e-87cc-cab454029efe for instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 on hv-cinder-maste Jul 05 10:18:29.345820 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-64167304-1445-4625-9b65-1c26cf0fcff8 req-45826a47-e91d-4935-a3b5-393327601a6d service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051580 Jul 05 10:18:29.350416 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 387/746] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:18:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:18:29.884037 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8466cd26-c66c-42a0-a698-11d33de1afb3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:29.885312 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-8466cd26-c66c-42a0-a698-11d33de1afb3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:29.899672 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8466cd26-c66c-42a0-a698-11d33de1afb3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:29.900384 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8466cd26-c66c-42a0-a698-11d33de1afb3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:29.902313 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8466cd26-c66c-42a0-a698-11d33de1afb3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:29.902916 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8466cd26-c66c-42a0-a698-11d33de1afb3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:30.015760 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8466cd26-c66c-42a0-a698-11d33de1afb3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:30.025534 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8466cd26-c66c-42a0-a698-11d33de1afb3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:30.028536 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8466cd26-c66c-42a0-a698-11d33de1afb3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:30.029986 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-8466cd26-c66c-42a0-a698-11d33de1afb3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1339 microversion: 2.20 time: 0.150799 Jul 05 10:18:30.031374 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 360/747] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:29 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1339 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:18:30.491680 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-053d25e6-2b8b-44fd-8f9d-62fe08a7acd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:30.492284 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-053d25e6-2b8b-44fd-8f9d-62fe08a7acd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:30.505781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-053d25e6-2b8b-44fd-8f9d-62fe08a7acd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:30.513166 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-053d25e6-2b8b-44fd-8f9d-62fe08a7acd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:30.515194 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-053d25e6-2b8b-44fd-8f9d-62fe08a7acd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:30.517318 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-053d25e6-2b8b-44fd-8f9d-62fe08a7acd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:30.604186 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-3681d6c6-301f-4279-920c-3400636bcad5 req-01b4f764-6e3a-45ca-b4fa-e0d86234f01a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5377ac3e-4f33-4b05-bc85-80b0d7b653c4", "tag": "080193f7-c96f-450e-87cc-cab454029efe"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:18:30.617109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-3681d6c6-301f-4279-920c-3400636bcad5 req-01b4f764-6e3a-45ca-b4fa-e0d86234f01a service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:30.618016 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-3681d6c6-301f-4279-920c-3400636bcad5 req-01b4f764-6e3a-45ca-b4fa-e0d86234f01a service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:30.631563 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-3681d6c6-301f-4279-920c-3400636bcad5 req-01b4f764-6e3a-45ca-b4fa-e0d86234f01a service nova] Creating event network-changed:080193f7-c96f-450e-87cc-cab454029efe for instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 on hv-cinder-maste Jul 05 10:18:30.639059 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-3681d6c6-301f-4279-920c-3400636bcad5 req-01b4f764-6e3a-45ca-b4fa-e0d86234f01a service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.148752 Jul 05 10:18:30.639983 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 361/748] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:18:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:18:30.640982 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.objects.instance [None req-053d25e6-2b8b-44fd-8f9d-62fe08a7acd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lazy-loading 'services' on Instance uuid 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 {{(pid=117813) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:18:30.720501 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-053d25e6-2b8b-44fd-8f9d-62fe08a7acd4 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "DELETE /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/os-volume_attachments/69d67d51-ed91-43a9-9f0e-2c5c195d34c6" status: 202 len: 0 microversion: 2.1 time: 0.233029 Jul 05 10:18:30.732321 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fee8950e-8906-4e52-8d3d-c67c82f9d68d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:30.734167 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fee8950e-8906-4e52-8d3d-c67c82f9d68d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:30.737103 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 388/749] 10.222.0.20 () {58 vars in 1415 bytes} [Tue Jul 5 10:18:30 2022] DELETE /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/os-volume_attachments/69d67d51-ed91-43a9-9f0e-2c5c195d34c6 => generated 0 bytes in 238 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:18:30.742381 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fee8950e-8906-4e52-8d3d-c67c82f9d68d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:30.743159 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fee8950e-8906-4e52-8d3d-c67c82f9d68d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:30.745040 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fee8950e-8906-4e52-8d3d-c67c82f9d68d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:30.746311 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fee8950e-8906-4e52-8d3d-c67c82f9d68d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:30.786503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fee8950e-8906-4e52-8d3d-c67c82f9d68d tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059463 Jul 05 10:18:30.791787 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 362/750] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:18:30 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:18:31.050037 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-05c63658-d328-438f-b40d-8ea7eea3d9a8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:31.050653 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-05c63658-d328-438f-b40d-8ea7eea3d9a8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:31.057183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-05c63658-d328-438f-b40d-8ea7eea3d9a8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:31.057629 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-05c63658-d328-438f-b40d-8ea7eea3d9a8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:31.058997 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-05c63658-d328-438f-b40d-8ea7eea3d9a8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:31.059425 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-05c63658-d328-438f-b40d-8ea7eea3d9a8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:31.192620 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-05c63658-d328-438f-b40d-8ea7eea3d9a8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:31.200541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-05c63658-d328-438f-b40d-8ea7eea3d9a8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:31.202020 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-05c63658-d328-438f-b40d-8ea7eea3d9a8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:31.203110 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-05c63658-d328-438f-b40d-8ea7eea3d9a8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1407 microversion: 2.20 time: 0.157513 Jul 05 10:18:31.204166 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 389/751] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:31 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1407 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 05 10:18:31.799285 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7288a976-f21b-43d2-ad48-868a1407fd83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:31.800243 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7288a976-f21b-43d2-ad48-868a1407fd83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:31.806721 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7288a976-f21b-43d2-ad48-868a1407fd83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:31.807340 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7288a976-f21b-43d2-ad48-868a1407fd83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:31.808137 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7288a976-f21b-43d2-ad48-868a1407fd83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:31.808613 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7288a976-f21b-43d2-ad48-868a1407fd83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:31.837149 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7288a976-f21b-43d2-ad48-868a1407fd83 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041795 Jul 05 10:18:31.837722 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 363/752] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:18:31 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 05 10:18:31.848366 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-951ff415-7371-42d1-bb59-f7cc60e7a352 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:31.848947 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-951ff415-7371-42d1-bb59-f7cc60e7a352 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:31.855574 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-951ff415-7371-42d1-bb59-f7cc60e7a352 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:31.855958 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-951ff415-7371-42d1-bb59-f7cc60e7a352 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:31.856870 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-951ff415-7371-42d1-bb59-f7cc60e7a352 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:31.857358 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-951ff415-7371-42d1-bb59-f7cc60e7a352 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:31.943180 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-951ff415-7371-42d1-bb59-f7cc60e7a352 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] HTTP exception thrown: Instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 is not attached to volume 69d67d51-ed91-43a9-9f0e-2c5c195d34c6 Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-951ff415-7371-42d1-bb59-f7cc60e7a352 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Request method failure captured: Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: DELETE /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/os-volume_attachments/69d67d51-ed91-43a9-9f0e-2c5c195d34c6 HTTP/1.1 Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA7oJUekaB5SIRB7laODsgwnK0t7jiVxN1Ai8dGBgn48BMKHLQSIe6dgIEAfEli2tjrGcZoyFrCcma0Vo9OV1uEdw7KPyLz_kc-21m2g94toBsR5yX2DPzbXnLhoIACHo9jqZb6Ax-rYpWgLn3COMUGLXvktG9fjU9_ROwdnh_YmvKk Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: d71ebc02c6094919b3054d7da84aadf0 Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-TestMinimumBasicScenario-376968780 Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:18:31.946030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:18:31.947548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/d71ebc02c6094919b3054d7da84aadf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_d71ebc02c6094919b3054d7da84aadf0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d71ebc02c6094919b3054d7da84aadf0"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d71ebc02c6094919b3054d7da84aadf0"}]}] Jul 05 10:18:31.947548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-TestMinimumBasicScenario-376968780 Jul 05 10:18:31.947548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: d71ebc02c6094919b3054d7da84aadf0 Jul 05 10:18:31.947548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-TestMinimumBasicScenario-376968780 Jul 05 10:18:31.947548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-TestMinimumBasicScenario-376968780-project Jul 05 10:18:31.947548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:18:31.947548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:18:31.947548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: f0351b9ab5ac4fbaad38df073518412a Jul 05 10:18:31.947548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-TestMinimumBasicScenario-376968780-project Jul 05 10:18:31.947548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:18:31.947548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 is not attached to volume 69d67d51-ed91-43a9-9f0e-2c5c195d34c6 Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'server_id': '5377ac3e-4f33-4b05-bc85-80b0d7b653c4', 'id': '69d67d51-ed91-43a9-9f0e-2c5c195d34c6'} Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 69d67d51-ed91-43a9-9f0e-2c5c195d34c6. Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:18:31.949108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 is not attached to volume 69d67d51-ed91-43a9-9f0e-2c5c195d34c6 Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 is not attached to volume 69d67d51-ed91-43a9-9f0e-2c5c195d34c6 Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:18:31.950409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-951ff415-7371-42d1-bb59-f7cc60e7a352 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Returning 404 to user: Instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 is not attached to volume 69d67d51-ed91-43a9-9f0e-2c5c195d34c6 {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:18:31.951355 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-951ff415-7371-42d1-bb59-f7cc60e7a352 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "DELETE /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/os-volume_attachments/69d67d51-ed91-43a9-9f0e-2c5c195d34c6" status: 404 len: 154 microversion: 2.1 time: 0.106424 Jul 05 10:18:31.951678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 390/753] 10.222.0.20 () {58 vars in 1415 bytes} [Tue Jul 5 10:18:31 2022] DELETE /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4/os-volume_attachments/69d67d51-ed91-43a9-9f0e-2c5c195d34c6 => generated 154 bytes in 108 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:18:32.161246 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-226b48b6-9d8f-4aa5-bb49-272027140bb8 req-cf3d02aa-8220-460f-8824-1a69481c4065 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ff442846-3ded-48c4-8a48-df125f46d0d6", "tag": "e6fb0593-584d-4308-86f5-4705f3dc20c8"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:18:32.175370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-226b48b6-9d8f-4aa5-bb49-272027140bb8 req-cf3d02aa-8220-460f-8824-1a69481c4065 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:32.175816 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-226b48b6-9d8f-4aa5-bb49-272027140bb8 req-cf3d02aa-8220-460f-8824-1a69481c4065 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:32.189688 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-226b48b6-9d8f-4aa5-bb49-272027140bb8 req-cf3d02aa-8220-460f-8824-1a69481c4065 service nova] Creating event network-changed:e6fb0593-584d-4308-86f5-4705f3dc20c8 for instance ff442846-3ded-48c4-8a48-df125f46d0d6 on hv-cinder-maste Jul 05 10:18:32.194352 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-226b48b6-9d8f-4aa5-bb49-272027140bb8 req-cf3d02aa-8220-460f-8824-1a69481c4065 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039915 Jul 05 10:18:32.197942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 364/754] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:18:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:18:32.221530 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d3226553-6b00-4e2e-bd66-c506b944850c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:32.222257 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d3226553-6b00-4e2e-bd66-c506b944850c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:32.228848 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d3226553-6b00-4e2e-bd66-c506b944850c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:32.229366 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d3226553-6b00-4e2e-bd66-c506b944850c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:32.230536 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d3226553-6b00-4e2e-bd66-c506b944850c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:32.230971 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d3226553-6b00-4e2e-bd66-c506b944850c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:32.465708 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d3226553-6b00-4e2e-bd66-c506b944850c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:32.473676 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d3226553-6b00-4e2e-bd66-c506b944850c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:32.475191 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d3226553-6b00-4e2e-bd66-c506b944850c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:32.476280 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d3226553-6b00-4e2e-bd66-c506b944850c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1407 microversion: 2.20 time: 0.258693 Jul 05 10:18:32.476857 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 391/755] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:32 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1407 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:18:33.104606 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2ea03331-8fb6-49c2-b458-51673f2c6f97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:33.105818 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2ea03331-8fb6-49c2-b458-51673f2c6f97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:33.112132 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2ea03331-8fb6-49c2-b458-51673f2c6f97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:33.112621 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2ea03331-8fb6-49c2-b458-51673f2c6f97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:33.113371 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2ea03331-8fb6-49c2-b458-51673f2c6f97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:33.113800 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2ea03331-8fb6-49c2-b458-51673f2c6f97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:33.137399 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2ea03331-8fb6-49c2-b458-51673f2c6f97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Going to try to terminate instance {{(pid=117812) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:18:33.208014 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2ea03331-8fb6-49c2-b458-51673f2c6f97 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "DELETE /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 204 len: 0 microversion: 2.1 time: 0.107486 Jul 05 10:18:33.208730 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 365/756] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:18:33 2022] DELETE /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:18:33.217205 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ae50f6e1-97f9-4ed5-a808-08aebf4778a3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:33.217813 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ae50f6e1-97f9-4ed5-a808-08aebf4778a3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:33.224194 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae50f6e1-97f9-4ed5-a808-08aebf4778a3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:33.224684 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae50f6e1-97f9-4ed5-a808-08aebf4778a3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:33.225441 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae50f6e1-97f9-4ed5-a808-08aebf4778a3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:33.225876 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ae50f6e1-97f9-4ed5-a808-08aebf4778a3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:33.354342 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ae50f6e1-97f9-4ed5-a808-08aebf4778a3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:33.361977 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ae50f6e1-97f9-4ed5-a808-08aebf4778a3 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1553 microversion: 2.1 time: 0.149178 Jul 05 10:18:33.362532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 392/757] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:33 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1553 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:33.495551 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-635d07b1-fe30-4c07-906c-5fc3695b32f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:33.496158 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-635d07b1-fe30-4c07-906c-5fc3695b32f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:33.502102 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-635d07b1-fe30-4c07-906c-5fc3695b32f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:33.502549 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-635d07b1-fe30-4c07-906c-5fc3695b32f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:33.503304 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-635d07b1-fe30-4c07-906c-5fc3695b32f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:33.503737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-635d07b1-fe30-4c07-906c-5fc3695b32f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:33.634639 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-635d07b1-fe30-4c07-906c-5fc3695b32f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:33.643072 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-635d07b1-fe30-4c07-906c-5fc3695b32f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:33.644590 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-635d07b1-fe30-4c07-906c-5fc3695b32f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:33.645678 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-635d07b1-fe30-4c07-906c-5fc3695b32f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1407 microversion: 2.20 time: 0.154461 Jul 05 10:18:33.646235 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 366/758] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:33 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1407 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:18:34.378903 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5883de22-b18f-455a-8b9f-afef17abae22 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:34.379505 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5883de22-b18f-455a-8b9f-afef17abae22 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:34.385916 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5883de22-b18f-455a-8b9f-afef17abae22 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:34.386358 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5883de22-b18f-455a-8b9f-afef17abae22 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:34.387109 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5883de22-b18f-455a-8b9f-afef17abae22 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:34.387544 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5883de22-b18f-455a-8b9f-afef17abae22 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:34.512346 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5883de22-b18f-455a-8b9f-afef17abae22 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:34.519642 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5883de22-b18f-455a-8b9f-afef17abae22 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1553 microversion: 2.1 time: 0.144517 Jul 05 10:18:34.520200 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 393/759] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:34 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1553 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:34.663294 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c374fa7c-12b0-403d-9aad-3d25c0964d37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:34.663832 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-c374fa7c-12b0-403d-9aad-3d25c0964d37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:34.670844 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c374fa7c-12b0-403d-9aad-3d25c0964d37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:34.671294 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c374fa7c-12b0-403d-9aad-3d25c0964d37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:34.672049 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c374fa7c-12b0-403d-9aad-3d25c0964d37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:34.672516 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c374fa7c-12b0-403d-9aad-3d25c0964d37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:34.804257 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-c374fa7c-12b0-403d-9aad-3d25c0964d37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:34.812005 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-c374fa7c-12b0-403d-9aad-3d25c0964d37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:34.813609 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-c374fa7c-12b0-403d-9aad-3d25c0964d37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:34.814709 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-c374fa7c-12b0-403d-9aad-3d25c0964d37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1407 microversion: 2.20 time: 0.155367 Jul 05 10:18:34.815261 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 367/760] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:34 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1407 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:18:35.536983 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2bfa5dbf-11a6-4654-894f-7153a00f31bf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:35.537514 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2bfa5dbf-11a6-4654-894f-7153a00f31bf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:35.544576 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2bfa5dbf-11a6-4654-894f-7153a00f31bf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:35.545026 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2bfa5dbf-11a6-4654-894f-7153a00f31bf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:35.545786 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2bfa5dbf-11a6-4654-894f-7153a00f31bf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:35.546221 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2bfa5dbf-11a6-4654-894f-7153a00f31bf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:35.672831 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2bfa5dbf-11a6-4654-894f-7153a00f31bf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:35.680231 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2bfa5dbf-11a6-4654-894f-7153a00f31bf tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1553 microversion: 2.1 time: 0.147152 Jul 05 10:18:35.680827 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 394/761] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:35 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1553 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:35.832535 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-be606cc1-abe2-4d44-99a1-fe6d5f368081 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:35.833070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-be606cc1-abe2-4d44-99a1-fe6d5f368081 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:35.840142 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be606cc1-abe2-4d44-99a1-fe6d5f368081 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:35.840637 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be606cc1-abe2-4d44-99a1-fe6d5f368081 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:35.841399 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be606cc1-abe2-4d44-99a1-fe6d5f368081 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:35.841836 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be606cc1-abe2-4d44-99a1-fe6d5f368081 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:35.973161 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-be606cc1-abe2-4d44-99a1-fe6d5f368081 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:35.980951 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-be606cc1-abe2-4d44-99a1-fe6d5f368081 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:35.982522 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-be606cc1-abe2-4d44-99a1-fe6d5f368081 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:35.983882 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-be606cc1-abe2-4d44-99a1-fe6d5f368081 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1407 microversion: 2.20 time: 0.155332 Jul 05 10:18:35.984460 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 368/762] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:35 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1407 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:18:36.386905 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:18:36.387076 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Tue Jul 5 10:18:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:18:36.697517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ab285880-6ba0-4bcd-b06d-ca26b4dfe650 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:36.698124 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ab285880-6ba0-4bcd-b06d-ca26b4dfe650 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:36.705205 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ab285880-6ba0-4bcd-b06d-ca26b4dfe650 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:36.705651 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ab285880-6ba0-4bcd-b06d-ca26b4dfe650 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:36.706415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ab285880-6ba0-4bcd-b06d-ca26b4dfe650 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:36.706852 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ab285880-6ba0-4bcd-b06d-ca26b4dfe650 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:36.878453 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ab285880-6ba0-4bcd-b06d-ca26b4dfe650 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:36.885882 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ab285880-6ba0-4bcd-b06d-ca26b4dfe650 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1553 microversion: 2.1 time: 0.192196 Jul 05 10:18:36.887042 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 395/763] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:36 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1553 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:37.001554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a66f0b4b-f50f-45c0-87b2-dafb635b2d61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:37.002154 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a66f0b4b-f50f-45c0-87b2-dafb635b2d61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:37.008627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a66f0b4b-f50f-45c0-87b2-dafb635b2d61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:37.009078 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a66f0b4b-f50f-45c0-87b2-dafb635b2d61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:37.009846 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a66f0b4b-f50f-45c0-87b2-dafb635b2d61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:37.010286 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a66f0b4b-f50f-45c0-87b2-dafb635b2d61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:37.153382 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a66f0b4b-f50f-45c0-87b2-dafb635b2d61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:37.161099 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a66f0b4b-f50f-45c0-87b2-dafb635b2d61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:37.162634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a66f0b4b-f50f-45c0-87b2-dafb635b2d61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:37.163725 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a66f0b4b-f50f-45c0-87b2-dafb635b2d61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1407 microversion: 2.20 time: 0.166071 Jul 05 10:18:37.164280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 369/764] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:36 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1407 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:18:37.726965 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-85d111ea-6099-425c-ab0c-ad5100c4708c req-c0ec265e-60e4-428c-8035-71ad23118a51 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5377ac3e-4f33-4b05-bc85-80b0d7b653c4", "name": "network-vif-deleted", "tag": "080193f7-c96f-450e-87cc-cab454029efe"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:18:37.731639 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=6 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:37.738366 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-85d111ea-6099-425c-ab0c-ad5100c4708c req-c0ec265e-60e4-428c-8035-71ad23118a51 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:37.738809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-85d111ea-6099-425c-ab0c-ad5100c4708c req-c0ec265e-60e4-428c-8035-71ad23118a51 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:37.750998 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-85d111ea-6099-425c-ab0c-ad5100c4708c req-c0ec265e-60e4-428c-8035-71ad23118a51 service nova] Creating event network-vif-deleted:080193f7-c96f-450e-87cc-cab454029efe for instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 on hv-cinder-maste Jul 05 10:18:37.756110 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-85d111ea-6099-425c-ab0c-ad5100c4708c req-c0ec265e-60e4-428c-8035-71ad23118a51 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032740 Jul 05 10:18:37.756857 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 396/765] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:18:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:18:37.903766 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-85742622-9b6b-4524-8fbe-71f7a6889e93 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:37.904484 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-85742622-9b6b-4524-8fbe-71f7a6889e93 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:37.906431 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:37.907079 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=6 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:37.913408 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-85742622-9b6b-4524-8fbe-71f7a6889e93 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:37.913851 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-85742622-9b6b-4524-8fbe-71f7a6889e93 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:37.914614 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-85742622-9b6b-4524-8fbe-71f7a6889e93 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:37.915098 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-85742622-9b6b-4524-8fbe-71f7a6889e93 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:37.957789 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-85742622-9b6b-4524-8fbe-71f7a6889e93 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:37.958157 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-85742622-9b6b-4524-8fbe-71f7a6889e93 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:37.958597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-85742622-9b6b-4524-8fbe-71f7a6889e93 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lazy-loading 'fault' on Instance uuid 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:18:37.999528 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-85742622-9b6b-4524-8fbe-71f7a6889e93 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f0351b9ab5ac4fbaad38df073518412a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd71ebc02c6094919b3054d7da84aadf0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:38.007641 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-85742622-9b6b-4524-8fbe-71f7a6889e93 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 200 len: 1347 microversion: 2.1 time: 0.108417 Jul 05 10:18:38.017666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 370/766] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:37 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 1347 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:18:38.182684 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-33c41597-c637-4674-8aa8-e4a70878b673 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:38.183939 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-33c41597-c637-4674-8aa8-e4a70878b673 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:38.190293 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-33c41597-c637-4674-8aa8-e4a70878b673 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:38.190750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-33c41597-c637-4674-8aa8-e4a70878b673 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:38.191515 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-33c41597-c637-4674-8aa8-e4a70878b673 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:38.191960 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-33c41597-c637-4674-8aa8-e4a70878b673 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:38.336754 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-33c41597-c637-4674-8aa8-e4a70878b673 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:38.344719 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-33c41597-c637-4674-8aa8-e4a70878b673 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:38.346232 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-33c41597-c637-4674-8aa8-e4a70878b673 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:38.347324 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-33c41597-c637-4674-8aa8-e4a70878b673 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1407 microversion: 2.20 time: 0.168875 Jul 05 10:18:38.347880 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 397/767] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:38 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1407 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:18:39.025211 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0094dc9b-318f-43f9-a4e6-81ed4496e231 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:39.025853 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0094dc9b-318f-43f9-a4e6-81ed4496e231 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] [instance: 5377ac3e-4f33-4b05-bc85-80b0d7b653c4] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:39.032737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0094dc9b-318f-43f9-a4e6-81ed4496e231 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:39.033183 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0094dc9b-318f-43f9-a4e6-81ed4496e231 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:39.033932 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0094dc9b-318f-43f9-a4e6-81ed4496e231 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:39.034475 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0094dc9b-318f-43f9-a4e6-81ed4496e231 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:39.048338 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-0094dc9b-318f-43f9-a4e6-81ed4496e231 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] HTTP exception thrown: Instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 could not be found. Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0094dc9b-318f-43f9-a4e6-81ed4496e231 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Request method failure captured: Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 HTTP/1.1 Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA7oJUekaB5SIRB7laODsgwnK0t7jiVxN1Ai8dGBgn48BMKHLQSIe6dgIEAfEli2tjrGcZoyFrCcma0Vo9OV1uEdw7KPyLz_kc-21m2g94toBsR5yX2DPzbXnLhoIACHo9jqZb6Ax-rYpWgLn3COMUGLXvktG9fjU9_ROwdnh_YmvKk Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: d71ebc02c6094919b3054d7da84aadf0 Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-TestMinimumBasicScenario-376968780 Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:18:39.049991 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:18:39.051469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/d71ebc02c6094919b3054d7da84aadf0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_d71ebc02c6094919b3054d7da84aadf0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d71ebc02c6094919b3054d7da84aadf0"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/d71ebc02c6094919b3054d7da84aadf0"}]}] Jul 05 10:18:39.051469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-TestMinimumBasicScenario-376968780 Jul 05 10:18:39.051469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: d71ebc02c6094919b3054d7da84aadf0 Jul 05 10:18:39.051469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-TestMinimumBasicScenario-376968780 Jul 05 10:18:39.051469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-TestMinimumBasicScenario-376968780-project Jul 05 10:18:39.051469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:18:39.051469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:18:39.051469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: f0351b9ab5ac4fbaad38df073518412a Jul 05 10:18:39.051469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-TestMinimumBasicScenario-376968780-project Jul 05 10:18:39.051469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:18:39.051469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 could not be found. Jul 05 10:18:39.051469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': '5377ac3e-4f33-4b05-bc85-80b0d7b653c4'} Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 could not be found. Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:18:39.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 could not be found. Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 could not be found. Jul 05 10:18:39.053808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:18:39.055159 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0094dc9b-318f-43f9-a4e6-81ed4496e231 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Returning 404 to user: Instance 5377ac3e-4f33-4b05-bc85-80b0d7b653c4 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:18:39.055159 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0094dc9b-318f-43f9-a4e6-81ed4496e231 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4" status: 404 len: 111 microversion: 2.1 time: 0.030080 Jul 05 10:18:39.055159 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 371/768] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:18:39 2022] GET /compute/v2.1/servers/5377ac3e-4f33-4b05-bc85-80b0d7b653c4 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:18:39.059908 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-62ec7461-5503-4442-bdb2-6700946e5865 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] Calling method '.version_select at 0x7feb896b2ca0>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:39.078978 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-62ec7461-5503-4442-bdb2-6700946e5865 tempest-TestMinimumBasicScenario-376968780 tempest-TestMinimumBasicScenario-376968780-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-934834031" status: 202 len: 0 microversion: 2.1 time: 0.022862 Jul 05 10:18:39.079532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 398/769] 10.222.0.20 () {58 vars in 1268 bytes} [Tue Jul 5 10:18:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-934834031 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:18:39.365334 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-1957d8f4-f5a2-421d-9eae-59cb871ddd63 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:39.366118 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-1957d8f4-f5a2-421d-9eae-59cb871ddd63 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:39.373402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1957d8f4-f5a2-421d-9eae-59cb871ddd63 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:39.373601 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1957d8f4-f5a2-421d-9eae-59cb871ddd63 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:39.374384 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1957d8f4-f5a2-421d-9eae-59cb871ddd63 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:39.374814 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1957d8f4-f5a2-421d-9eae-59cb871ddd63 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:39.512283 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1957d8f4-f5a2-421d-9eae-59cb871ddd63 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:39.520329 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1957d8f4-f5a2-421d-9eae-59cb871ddd63 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:39.521878 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1957d8f4-f5a2-421d-9eae-59cb871ddd63 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:39.522975 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-1957d8f4-f5a2-421d-9eae-59cb871ddd63 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1407 microversion: 2.20 time: 0.161862 Jul 05 10:18:39.523524 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 372/770] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:39 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1407 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:18:39.799401 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-059c51c3-b143-4cc9-bf18-ed1c56a9026b req-06f6e8f5-4d13-4021-9aae-b51808759320 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ff442846-3ded-48c4-8a48-df125f46d0d6", "name": "network-vif-plugged", "status": "completed", "tag": "e6fb0593-584d-4308-86f5-4705f3dc20c8"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:18:39.811002 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-059c51c3-b143-4cc9-bf18-ed1c56a9026b req-06f6e8f5-4d13-4021-9aae-b51808759320 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:39.811557 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-059c51c3-b143-4cc9-bf18-ed1c56a9026b req-06f6e8f5-4d13-4021-9aae-b51808759320 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:39.824543 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-059c51c3-b143-4cc9-bf18-ed1c56a9026b req-06f6e8f5-4d13-4021-9aae-b51808759320 service nova] Creating event network-vif-plugged:e6fb0593-584d-4308-86f5-4705f3dc20c8 for instance ff442846-3ded-48c4-8a48-df125f46d0d6 on hv-cinder-maste Jul 05 10:18:39.829636 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-059c51c3-b143-4cc9-bf18-ed1c56a9026b req-06f6e8f5-4d13-4021-9aae-b51808759320 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034167 Jul 05 10:18:39.831104 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 399/771] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:18:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:18:40.540780 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-002b2d81-7e23-44d8-8619-246b51748695 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:40.541490 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-002b2d81-7e23-44d8-8619-246b51748695 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:40.548510 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-002b2d81-7e23-44d8-8619-246b51748695 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:40.548985 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-002b2d81-7e23-44d8-8619-246b51748695 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:40.549742 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-002b2d81-7e23-44d8-8619-246b51748695 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:40.550176 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-002b2d81-7e23-44d8-8619-246b51748695 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:40.680347 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-002b2d81-7e23-44d8-8619-246b51748695 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:40.688550 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-002b2d81-7e23-44d8-8619-246b51748695 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:40.690056 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-002b2d81-7e23-44d8-8619-246b51748695 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:40.691240 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-002b2d81-7e23-44d8-8619-246b51748695 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1407 microversion: 2.20 time: 0.154409 Jul 05 10:18:40.691787 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 373/772] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:40 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1407 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:18:41.710469 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ed2113d7-0760-42db-b632-79ee6ff39c1a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:41.711303 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ed2113d7-0760-42db-b632-79ee6ff39c1a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:41.717690 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ed2113d7-0760-42db-b632-79ee6ff39c1a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:41.718299 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ed2113d7-0760-42db-b632-79ee6ff39c1a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:41.719151 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ed2113d7-0760-42db-b632-79ee6ff39c1a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:41.719680 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ed2113d7-0760-42db-b632-79ee6ff39c1a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:41.884642 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ed2113d7-0760-42db-b632-79ee6ff39c1a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:41.893790 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ed2113d7-0760-42db-b632-79ee6ff39c1a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:41.895790 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ed2113d7-0760-42db-b632-79ee6ff39c1a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:41.896450 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ed2113d7-0760-42db-b632-79ee6ff39c1a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1407 microversion: 2.20 time: 0.191251 Jul 05 10:18:41.897589 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 400/773] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:41 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1407 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:18:42.914625 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a631928e-2e1b-4f35-bffd-71559b7e8a13 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:42.915249 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a631928e-2e1b-4f35-bffd-71559b7e8a13 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:42.923768 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a631928e-2e1b-4f35-bffd-71559b7e8a13 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:42.924222 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a631928e-2e1b-4f35-bffd-71559b7e8a13 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:42.925030 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a631928e-2e1b-4f35-bffd-71559b7e8a13 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:42.925518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a631928e-2e1b-4f35-bffd-71559b7e8a13 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:43.076937 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a631928e-2e1b-4f35-bffd-71559b7e8a13 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:43.084915 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a631928e-2e1b-4f35-bffd-71559b7e8a13 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:43.086732 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a631928e-2e1b-4f35-bffd-71559b7e8a13 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:18:43.088685 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a631928e-2e1b-4f35-bffd-71559b7e8a13 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1599 microversion: 2.20 time: 0.177943 Jul 05 10:18:43.090368 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 374/774] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:18:42 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1599 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:18:43.109892 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-92b7f7d8-af94-4cb3-8024-e2dbea6b6c56 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:18:43.110377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-92b7f7d8-af94-4cb3-8024-e2dbea6b6c56 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:18:43.116741 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-92b7f7d8-af94-4cb3-8024-e2dbea6b6c56 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:43.117185 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-92b7f7d8-af94-4cb3-8024-e2dbea6b6c56 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:43.117940 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-92b7f7d8-af94-4cb3-8024-e2dbea6b6c56 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:43.118366 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-92b7f7d8-af94-4cb3-8024-e2dbea6b6c56 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:43.199865 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-92b7f7d8-af94-4cb3-8024-e2dbea6b6c56 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.094040 Jul 05 10:18:43.200600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 401/775] 10.222.0.20 () {60 vars in 1315 bytes} [Tue Jul 5 10:18:43 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-interface => generated 284 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:18:44.426336 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-db7b0fff-aca8-48f2-8b71-c7d39e266f7e req-08517367-8bdf-4395-b250-30b8b0f033c4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ff442846-3ded-48c4-8a48-df125f46d0d6", "tag": "e6fb0593-584d-4308-86f5-4705f3dc20c8"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:18:44.434372 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:18:44.436823 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-db7b0fff-aca8-48f2-8b71-c7d39e266f7e req-08517367-8bdf-4395-b250-30b8b0f033c4 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:18:44.437507 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-db7b0fff-aca8-48f2-8b71-c7d39e266f7e req-08517367-8bdf-4395-b250-30b8b0f033c4 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:18:44.452711 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-db7b0fff-aca8-48f2-8b71-c7d39e266f7e req-08517367-8bdf-4395-b250-30b8b0f033c4 service nova] Creating event network-changed:e6fb0593-584d-4308-86f5-4705f3dc20c8 for instance ff442846-3ded-48c4-8a48-df125f46d0d6 on hv-cinder-maste Jul 05 10:18:44.458716 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-db7b0fff-aca8-48f2-8b71-c7d39e266f7e req-08517367-8bdf-4395-b250-30b8b0f033c4 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035918 Jul 05 10:18:44.463789 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 375/776] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:18:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:19:02.704209 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=55,nova_api:INSERT=1,nova_api:DELETE=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:02.704992 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=79,nova_cell1:INSERT=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:02.709422 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-7d2519e1-1b8a-42e5-bd78-2523e425b414 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:02.709913 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7d2519e1-1b8a-42e5-bd78-2523e425b414 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:02.717606 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7d2519e1-1b8a-42e5-bd78-2523e425b414 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:02.718062 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7d2519e1-1b8a-42e5-bd78-2523e425b414 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:02.718830 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7d2519e1-1b8a-42e5-bd78-2523e425b414 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:02.719272 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7d2519e1-1b8a-42e5-bd78-2523e425b414 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:02.743521 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-7d2519e1-1b8a-42e5-bd78-2523e425b414 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/ips" status: 200 len: 149 microversion: 2.1 time: 0.040987 Jul 05 10:19:02.744083 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 402/777] 10.222.0.20 () {58 vars in 1247 bytes} [Tue Jul 5 10:19:02 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/ips => generated 149 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:19:04.051685 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e41aeed5-464a-44ce-8101-148370a99dfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bea65a97-8a07-442c-a72b-c1cdefe3e1cd"}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:19:04.052857 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e41aeed5-464a-44ce-8101-148370a99dfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:04.055479 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:04.062154 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e41aeed5-464a-44ce-8101-148370a99dfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:04.062610 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e41aeed5-464a-44ce-8101-148370a99dfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:04.063376 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e41aeed5-464a-44ce-8101-148370a99dfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:04.063814 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e41aeed5-464a-44ce-8101-148370a99dfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:04.317046 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e41aeed5-464a-44ce-8101-148370a99dfc tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "POST /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.269406 Jul 05 10:19:04.317669 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 376/778] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:19:04 2022] POST /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/os-volume_attachments => generated 194 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:19:11.797759 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-53512eae-cb73-43c9-bd40-ed47c5261ef8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bea65a97-8a07-442c-a72b-c1cdefe3e1cd"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:19:11.798808 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-53512eae-cb73-43c9-bd40-ed47c5261ef8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:11.805129 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-53512eae-cb73-43c9-bd40-ed47c5261ef8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:11.805570 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-53512eae-cb73-43c9-bd40-ed47c5261ef8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:11.806319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-53512eae-cb73-43c9-bd40-ed47c5261ef8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:11.806751 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-53512eae-cb73-43c9-bd40-ed47c5261ef8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:11.885023 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-53512eae-cb73-43c9-bd40-ed47c5261ef8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Invalid volume: volume bea65a97-8a07-442c-a72b-c1cdefe3e1cd is already attached to instances: 70cdd8c1-3058-4809-a0d1-73e39cf2e520 Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-53512eae-cb73-43c9-bd40-ed47c5261ef8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: POST /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/os-volume_attachments HTTP/1.1 Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 74 Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:19:11.886595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:19:11.888068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:19:11.888068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:19:11.888068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:19:11.888068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:19:11.888068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:19:11.888068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:19:11.888068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:19:11.888068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:19:11.888068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:19:11.888068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Jul 05 10:19:11.888068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {"volumeAttachment": {"volumeId": "bea65a97-8a07-442c-a72b-c1cdefe3e1cd"}} Jul 05 10:19:11.888068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Invalid volume: volume bea65a97-8a07-442c-a72b-c1cdefe3e1cd is already attached to instances: 70cdd8c1-3058-4809-a0d1-73e39cf2e520 Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'server_id': '70cdd8c1-3058-4809-a0d1-73e39cf2e520', 'body': {'volumeAttachment': {'volumeId': 'bea65a97-8a07-442c-a72b-c1cdefe3e1cd'}}} Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume bea65a97-8a07-442c-a72b-c1cdefe3e1cd is already attached to instances: 70cdd8c1-3058-4809-a0d1-73e39cf2e520 Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:19:11.888854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume bea65a97-8a07-442c-a72b-c1cdefe3e1cd is already attached to instances: 70cdd8c1-3058-4809-a0d1-73e39cf2e520 Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:19:11.890775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:19:11.892580 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:19:11.892580 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:19:11.892580 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:19:11.892580 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume bea65a97-8a07-442c-a72b-c1cdefe3e1cd is already attached to instances: 70cdd8c1-3058-4809-a0d1-73e39cf2e520 Jul 05 10:19:11.892580 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:19:11.892580 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-53512eae-cb73-43c9-bd40-ed47c5261ef8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 400 to user: Invalid volume: volume bea65a97-8a07-442c-a72b-c1cdefe3e1cd is already attached to instances: 70cdd8c1-3058-4809-a0d1-73e39cf2e520 {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:19:11.892580 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-53512eae-cb73-43c9-bd40-ed47c5261ef8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "POST /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.094187 Jul 05 10:19:11.892580 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 403/779] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:19:11 2022] POST /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/os-volume_attachments => generated 174 bytes in 95 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 05 10:19:11.949255 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-4f7138b3-01a3-4ddb-ac4b-c66501ac0bba tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:11.949775 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-4f7138b3-01a3-4ddb-ac4b-c66501ac0bba tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:11.956695 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4f7138b3-01a3-4ddb-ac4b-c66501ac0bba tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:11.957150 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4f7138b3-01a3-4ddb-ac4b-c66501ac0bba tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:11.957919 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4f7138b3-01a3-4ddb-ac4b-c66501ac0bba tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:11.958362 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4f7138b3-01a3-4ddb-ac4b-c66501ac0bba tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:12.038224 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-4f7138b3-01a3-4ddb-ac4b-c66501ac0bba tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lazy-loading 'services' on Instance uuid 70cdd8c1-3058-4809-a0d1-73e39cf2e520 {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:19:12.109547 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-4f7138b3-01a3-4ddb-ac4b-c66501ac0bba tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/os-volume_attachments/bea65a97-8a07-442c-a72b-c1cdefe3e1cd" status: 202 len: 0 microversion: 2.1 time: 0.165045 Jul 05 10:19:12.110170 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 377/780] 10.222.0.20 () {58 vars in 1415 bytes} [Tue Jul 5 10:19:11 2022] DELETE /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/os-volume_attachments/bea65a97-8a07-442c-a72b-c1cdefe3e1cd => generated 0 bytes in 167 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:19:12.118898 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-454b99e5-34f9-491f-8fbc-2e3247bb1063 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:12.119816 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-454b99e5-34f9-491f-8fbc-2e3247bb1063 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:12.125806 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-454b99e5-34f9-491f-8fbc-2e3247bb1063 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:12.126315 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-454b99e5-34f9-491f-8fbc-2e3247bb1063 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:12.127097 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-454b99e5-34f9-491f-8fbc-2e3247bb1063 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:12.127534 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-454b99e5-34f9-491f-8fbc-2e3247bb1063 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:12.157229 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-454b99e5-34f9-491f-8fbc-2e3247bb1063 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042193 Jul 05 10:19:12.157790 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 404/781] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:19:12 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:19:13.169889 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-82566a28-34fe-48b3-982f-35676aca7fd8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:13.171090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-82566a28-34fe-48b3-982f-35676aca7fd8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:13.177286 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-82566a28-34fe-48b3-982f-35676aca7fd8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:13.177739 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-82566a28-34fe-48b3-982f-35676aca7fd8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:13.178501 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-82566a28-34fe-48b3-982f-35676aca7fd8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:13.178936 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-82566a28-34fe-48b3-982f-35676aca7fd8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:13.206823 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-82566a28-34fe-48b3-982f-35676aca7fd8 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041685 Jul 05 10:19:13.207361 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 378/782] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:19:13 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 05 10:19:13.263517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-130e9c77-3278-4f47-ad80-c5813fa4554f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:13.263990 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-130e9c77-3278-4f47-ad80-c5813fa4554f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:13.270868 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-130e9c77-3278-4f47-ad80-c5813fa4554f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:13.271304 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-130e9c77-3278-4f47-ad80-c5813fa4554f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:13.272051 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-130e9c77-3278-4f47-ad80-c5813fa4554f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:13.272509 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-130e9c77-3278-4f47-ad80-c5813fa4554f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:13.296864 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-130e9c77-3278-4f47-ad80-c5813fa4554f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Going to try to terminate instance {{(pid=117813) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:19:13.369425 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-130e9c77-3278-4f47-ad80-c5813fa4554f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 204 len: 0 microversion: 2.1 time: 0.109689 Jul 05 10:19:13.370218 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 405/783] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:19:13 2022] DELETE /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:19:13.378983 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-de661161-df1c-41dc-b1c2-1febdbc75e3d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:13.379591 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-de661161-df1c-41dc-b1c2-1febdbc75e3d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:13.385485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-de661161-df1c-41dc-b1c2-1febdbc75e3d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:13.385934 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-de661161-df1c-41dc-b1c2-1febdbc75e3d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:13.386697 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-de661161-df1c-41dc-b1c2-1febdbc75e3d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:13.387135 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-de661161-df1c-41dc-b1c2-1febdbc75e3d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:13.514445 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-de661161-df1c-41dc-b1c2-1febdbc75e3d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:13.522396 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-de661161-df1c-41dc-b1c2-1febdbc75e3d tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 200 len: 1679 microversion: 2.1 time: 0.147150 Jul 05 10:19:13.522964 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 379/784] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:13 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 1679 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:14.539557 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-7af9a8f4-4eb1-4334-9309-a5c32403fe23 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:14.540242 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7af9a8f4-4eb1-4334-9309-a5c32403fe23 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:14.546671 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7af9a8f4-4eb1-4334-9309-a5c32403fe23 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:14.547119 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7af9a8f4-4eb1-4334-9309-a5c32403fe23 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:14.547872 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7af9a8f4-4eb1-4334-9309-a5c32403fe23 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:14.548344 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7af9a8f4-4eb1-4334-9309-a5c32403fe23 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:14.678578 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-7af9a8f4-4eb1-4334-9309-a5c32403fe23 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:14.685924 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-7af9a8f4-4eb1-4334-9309-a5c32403fe23 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 200 len: 1679 microversion: 2.1 time: 0.150583 Jul 05 10:19:14.686484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 406/785] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:14 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 1679 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:15.703718 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6ee3a58c-661d-4cc6-bb20-65057f8a640f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:15.704457 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6ee3a58c-661d-4cc6-bb20-65057f8a640f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:15.711032 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6ee3a58c-661d-4cc6-bb20-65057f8a640f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:15.711484 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6ee3a58c-661d-4cc6-bb20-65057f8a640f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:15.712266 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6ee3a58c-661d-4cc6-bb20-65057f8a640f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:15.712717 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6ee3a58c-661d-4cc6-bb20-65057f8a640f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:15.939873 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6ee3a58c-661d-4cc6-bb20-65057f8a640f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:15.947306 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6ee3a58c-661d-4cc6-bb20-65057f8a640f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 200 len: 1679 microversion: 2.1 time: 0.248139 Jul 05 10:19:15.947867 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 380/786] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:15 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 1679 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:16.157515 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-af1992b8-7c62-47ae-b17c-63e6098fbff3 req-16baf6cc-b180-435f-909b-5749335d7932 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "70cdd8c1-3058-4809-a0d1-73e39cf2e520", "name": "network-vif-deleted", "tag": "dbd17c56-2b84-42e7-9867-b2fcb6dc597e"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:19:16.168657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-af1992b8-7c62-47ae-b17c-63e6098fbff3 req-16baf6cc-b180-435f-909b-5749335d7932 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:16.169103 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-af1992b8-7c62-47ae-b17c-63e6098fbff3 req-16baf6cc-b180-435f-909b-5749335d7932 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:16.180418 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-af1992b8-7c62-47ae-b17c-63e6098fbff3 req-16baf6cc-b180-435f-909b-5749335d7932 service nova] Creating event network-vif-deleted:dbd17c56-2b84-42e7-9867-b2fcb6dc597e for instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 on hv-cinder-maste Jul 05 10:19:16.185332 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-af1992b8-7c62-47ae-b17c-63e6098fbff3 req-16baf6cc-b180-435f-909b-5749335d7932 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032253 Jul 05 10:19:16.186311 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 407/787] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:19:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:19:16.964864 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-850aa177-687b-4b91-bec7-5441560627bd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:16.965487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-850aa177-687b-4b91-bec7-5441560627bd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:16.972914 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-850aa177-687b-4b91-bec7-5441560627bd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:16.973370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-850aa177-687b-4b91-bec7-5441560627bd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:16.974157 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-850aa177-687b-4b91-bec7-5441560627bd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:16.974598 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-850aa177-687b-4b91-bec7-5441560627bd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:16.988544 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-850aa177-687b-4b91-bec7-5441560627bd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-850aa177-687b-4b91-bec7-5441560627bd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 HTTP/1.1 Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:19:16.990204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:19:16.991485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:19:16.991485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:19:16.991485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:19:16.991485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:19:16.991485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:19:16.991485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:19:16.991485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:19:16.991485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:19:16.991485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:19:16.991485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:19:16.991485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:19:16.991485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': '70cdd8c1-3058-4809-a0d1-73e39cf2e520'} Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:19:16.992219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:19:16.993496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:19:16.994735 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-850aa177-687b-4b91-bec7-5441560627bd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 404 to user: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:19:16.994735 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-850aa177-687b-4b91-bec7-5441560627bd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 404 len: 111 microversion: 2.1 time: 0.030658 Jul 05 10:19:16.994735 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 381/788] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:16 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:19:17.001033 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8ddf90d1-0542-476e-8193-68ad3a992bdf tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '.version_select at 0x7feb896b21f0>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:17.020311 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8ddf90d1-0542-476e-8193-68ad3a992bdf tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-973925554" status: 202 len: 0 microversion: 2.1 time: 0.023456 Jul 05 10:19:17.021078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 408/789] 10.222.0.20 () {58 vars in 1217 bytes} [Tue Jul 5 10:19:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-973925554 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:19:17.637508 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2edb3ecc-bffa-463e-9457-9eb06719a33b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Action: 'create', calling method: .version_select at 0x7feb895093a0>, body: {"keypair": {"name": "tempest-keypair-1306729611", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHgf0Nif0LIuS3P1/CEaggIL/pviMEeRCio352nZS6QM0ByBi82+L54MYPmjMAd5rniThq2NSl7wKu2sPVW8sM42VkxmEHqVGGTBzGkIjvhz1XPDYvENzsYFy+FT+TajEw=="}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:19:17.651402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-2edb3ecc-bffa-463e-9457-9eb06719a33b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:19:17.655187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-2edb3ecc-bffa-463e-9457-9eb06719a33b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for user 036cb887c7894e75ba3eb4c1a8b21c69 and project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:19:17.676503 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2edb3ecc-bffa-463e-9457-9eb06719a33b tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.042825 Jul 05 10:19:17.677083 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 382/790] 10.222.0.20 () {60 vars in 1158 bytes} [Tue Jul 5 10:19:17 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:19:18.919750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1169123718", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "033e3c55-aad3-40e5-8b8b-1dedb7b873c7"}], "security_groups": [{"name": "tempest-securitygroup--1825765942"}], "key_name": "tempest-keypair-1306729611", "user_data": "}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:19:19.022637 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.network.neutron [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] validate_networks() for [('033e3c55-aad3-40e5-8b8b-1dedb7b873c7', None, None, None, None, None)] {{(pid=117813) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:19:19.379376 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Flavor limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:19:19.379820 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Image limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:19:19.380283 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Flavor pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:19:19.380721 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Image pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:19:19.381153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:19:19.389827 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:19.390266 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:19.391047 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:19.391474 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:19.408019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:19:19.412035 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Getting quotas for user 036cb887c7894e75ba3eb4c1a8b21c69 and project 7eccead420ae4aacb6f31dcb17a138cc. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:19:19.423241 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Going to run 1 instances... {{(pid=117813) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:19:19.425549 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117813) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:19:19.470734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-765deced-9d29-43b5-ac2f-3ed7ba2d9515 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.555764 Jul 05 10:19:19.471479 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 409/791] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:19:18 2022] POST /compute/v2.1/servers => generated 402 bytes in 557 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:19:19.481899 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-268c3dcd-59df-4b41-9d8a-b23737c940aa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:19.482515 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-268c3dcd-59df-4b41-9d8a-b23737c940aa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:19.537310 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-268c3dcd-59df-4b41-9d8a-b23737c940aa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:19.544202 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-268c3dcd-59df-4b41-9d8a-b23737c940aa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1242 microversion: 2.1 time: 0.066399 Jul 05 10:19:19.544803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 383/792] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:19 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1242 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:20.562312 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ee7692f4-a9fb-4f84-bf86-b32151259941 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:20.562925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ee7692f4-a9fb-4f84-bf86-b32151259941 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:20.569816 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ee7692f4-a9fb-4f84-bf86-b32151259941 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:20.570297 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ee7692f4-a9fb-4f84-bf86-b32151259941 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:20.571058 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ee7692f4-a9fb-4f84-bf86-b32151259941 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:20.571497 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ee7692f4-a9fb-4f84-bf86-b32151259941 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:20.661680 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-ee7692f4-a9fb-4f84-bf86-b32151259941 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:20.669330 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ee7692f4-a9fb-4f84-bf86-b32151259941 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1300 microversion: 2.1 time: 0.111506 Jul 05 10:19:20.669904 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 410/793] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:20 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1300 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:21.686819 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-25c7bcc3-3bf8-4a7b-865a-cd9ec9f4d9b2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:21.687430 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-25c7bcc3-3bf8-4a7b-865a-cd9ec9f4d9b2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:21.693916 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-25c7bcc3-3bf8-4a7b-865a-cd9ec9f4d9b2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:21.694366 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-25c7bcc3-3bf8-4a7b-865a-cd9ec9f4d9b2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:21.695136 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-25c7bcc3-3bf8-4a7b-865a-cd9ec9f4d9b2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:21.695579 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-25c7bcc3-3bf8-4a7b-865a-cd9ec9f4d9b2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:21.920979 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-25c7bcc3-3bf8-4a7b-865a-cd9ec9f4d9b2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:21.931790 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-25c7bcc3-3bf8-4a7b-865a-cd9ec9f4d9b2 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1368 microversion: 2.1 time: 0.248880 Jul 05 10:19:21.932875 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 384/794] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:21 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1368 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:22.565323 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-89cd79b9-67a6-4c1f-b37b-70c33c74c0b3 req-376602da-a4f8-4eca-be89-383850ec32e5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d872dbf0-7f48-4887-85d4-9b9d0123be14", "tag": "705d8fd5-bec7-4c13-8533-3ab583c1b1bb"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:19:22.576101 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-89cd79b9-67a6-4c1f-b37b-70c33c74c0b3 req-376602da-a4f8-4eca-be89-383850ec32e5 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:22.576896 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-89cd79b9-67a6-4c1f-b37b-70c33c74c0b3 req-376602da-a4f8-4eca-be89-383850ec32e5 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:22.589818 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-89cd79b9-67a6-4c1f-b37b-70c33c74c0b3 req-376602da-a4f8-4eca-be89-383850ec32e5 service nova] Creating event network-changed:705d8fd5-bec7-4c13-8533-3ab583c1b1bb for instance d872dbf0-7f48-4887-85d4-9b9d0123be14 on hv-cinder-maste Jul 05 10:19:22.595700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-89cd79b9-67a6-4c1f-b37b-70c33c74c0b3 req-376602da-a4f8-4eca-be89-383850ec32e5 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034024 Jul 05 10:19:22.596251 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 411/795] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:19:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:19:22.949852 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8e02ab4e-c281-4021-8bf0-2194a38ad822 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:22.950474 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-8e02ab4e-c281-4021-8bf0-2194a38ad822 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:22.957470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8e02ab4e-c281-4021-8bf0-2194a38ad822 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:22.957911 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8e02ab4e-c281-4021-8bf0-2194a38ad822 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:22.958683 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8e02ab4e-c281-4021-8bf0-2194a38ad822 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:22.959127 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8e02ab4e-c281-4021-8bf0-2194a38ad822 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:23.097642 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8e02ab4e-c281-4021-8bf0-2194a38ad822 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:23.105165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-8e02ab4e-c281-4021-8bf0-2194a38ad822 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1368 microversion: 2.1 time: 0.159275 Jul 05 10:19:23.105725 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 385/796] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:22 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1368 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:24.122964 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-7840dd58-03c7-4357-b733-146ce8cfccfa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:24.123650 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7840dd58-03c7-4357-b733-146ce8cfccfa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:24.130387 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7840dd58-03c7-4357-b733-146ce8cfccfa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:24.130827 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7840dd58-03c7-4357-b733-146ce8cfccfa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:24.131574 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7840dd58-03c7-4357-b733-146ce8cfccfa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:24.132003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7840dd58-03c7-4357-b733-146ce8cfccfa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:24.259483 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-7840dd58-03c7-4357-b733-146ce8cfccfa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:24.267064 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-7840dd58-03c7-4357-b733-146ce8cfccfa tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1368 microversion: 2.1 time: 0.148528 Jul 05 10:19:24.267629 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 412/797] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:24 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1368 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:25.283628 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-23e2c977-1f70-459f-a72d-482478114adb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:25.284268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-23e2c977-1f70-459f-a72d-482478114adb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:25.290797 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-23e2c977-1f70-459f-a72d-482478114adb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:25.291250 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-23e2c977-1f70-459f-a72d-482478114adb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:25.292015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-23e2c977-1f70-459f-a72d-482478114adb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:25.292479 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-23e2c977-1f70-459f-a72d-482478114adb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:25.521155 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-23e2c977-1f70-459f-a72d-482478114adb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:25.528443 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-23e2c977-1f70-459f-a72d-482478114adb tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1368 microversion: 2.1 time: 0.249218 Jul 05 10:19:25.529012 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 386/798] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:25 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1368 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:26.545739 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-78c78f1e-a269-49fd-a1cc-61fc3e466109 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:26.546317 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-78c78f1e-a269-49fd-a1cc-61fc3e466109 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:26.552694 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-78c78f1e-a269-49fd-a1cc-61fc3e466109 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:26.553141 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-78c78f1e-a269-49fd-a1cc-61fc3e466109 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:26.553892 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-78c78f1e-a269-49fd-a1cc-61fc3e466109 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:26.554323 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-78c78f1e-a269-49fd-a1cc-61fc3e466109 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:26.681139 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-78c78f1e-a269-49fd-a1cc-61fc3e466109 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:26.689125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-78c78f1e-a269-49fd-a1cc-61fc3e466109 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1368 microversion: 2.1 time: 0.147171 Jul 05 10:19:26.689683 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 413/799] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:26 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1368 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:27.704941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5d9480fa-21ec-44f6-bad5-ce309a316f3f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:27.705543 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5d9480fa-21ec-44f6-bad5-ce309a316f3f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:27.712074 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5d9480fa-21ec-44f6-bad5-ce309a316f3f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:27.712559 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5d9480fa-21ec-44f6-bad5-ce309a316f3f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:27.713314 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5d9480fa-21ec-44f6-bad5-ce309a316f3f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:27.713749 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5d9480fa-21ec-44f6-bad5-ce309a316f3f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:27.841751 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5d9480fa-21ec-44f6-bad5-ce309a316f3f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:27.851124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5d9480fa-21ec-44f6-bad5-ce309a316f3f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1368 microversion: 2.1 time: 0.149916 Jul 05 10:19:27.851696 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 387/800] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:27 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1368 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:28.868753 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3d7b96b4-7216-4cbd-933b-84d1d45840a9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:28.869387 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3d7b96b4-7216-4cbd-933b-84d1d45840a9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:28.875789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3d7b96b4-7216-4cbd-933b-84d1d45840a9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:28.876229 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3d7b96b4-7216-4cbd-933b-84d1d45840a9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:28.877016 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3d7b96b4-7216-4cbd-933b-84d1d45840a9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:28.877446 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3d7b96b4-7216-4cbd-933b-84d1d45840a9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:29.011062 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-3d7b96b4-7216-4cbd-933b-84d1d45840a9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:29.018940 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3d7b96b4-7216-4cbd-933b-84d1d45840a9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1368 microversion: 2.1 time: 0.153986 Jul 05 10:19:29.019523 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 414/801] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:28 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1368 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:30.035581 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-128ceb79-1f42-43c3-8242-c3e9db920f42 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:30.036196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-128ceb79-1f42-43c3-8242-c3e9db920f42 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:30.038680 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=6 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:30.044059 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-128ceb79-1f42-43c3-8242-c3e9db920f42 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:30.044538 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-128ceb79-1f42-43c3-8242-c3e9db920f42 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:30.045312 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-128ceb79-1f42-43c3-8242-c3e9db920f42 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:30.045755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-128ceb79-1f42-43c3-8242-c3e9db920f42 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:30.059882 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-d696202f-3a7b-46f6-95f1-6e38ef146b1c req-a83b37aa-950b-48cb-bc98-eae03a4905a8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d872dbf0-7f48-4887-85d4-9b9d0123be14", "name": "network-vif-plugged", "status": "completed", "tag": "705d8fd5-bec7-4c13-8533-3ab583c1b1bb"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:19:30.064278 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:30.065012 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=6 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:30.074134 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-d696202f-3a7b-46f6-95f1-6e38ef146b1c req-a83b37aa-950b-48cb-bc98-eae03a4905a8 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:30.074784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-d696202f-3a7b-46f6-95f1-6e38ef146b1c req-a83b37aa-950b-48cb-bc98-eae03a4905a8 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:30.086620 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-d696202f-3a7b-46f6-95f1-6e38ef146b1c req-a83b37aa-950b-48cb-bc98-eae03a4905a8 service nova] Creating event network-vif-plugged:705d8fd5-bec7-4c13-8533-3ab583c1b1bb for instance d872dbf0-7f48-4887-85d4-9b9d0123be14 on hv-cinder-maste Jul 05 10:19:30.091685 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-d696202f-3a7b-46f6-95f1-6e38ef146b1c req-a83b37aa-950b-48cb-bc98-eae03a4905a8 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035468 Jul 05 10:19:30.092279 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 415/802] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:19:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:19:30.180194 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-128ceb79-1f42-43c3-8242-c3e9db920f42 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:30.189192 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-128ceb79-1f42-43c3-8242-c3e9db920f42 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1368 microversion: 2.1 time: 0.157433 Jul 05 10:19:30.189768 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 388/803] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:30 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1368 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:31.206652 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5bce3056-ce91-4a56-b763-6a336cd1aaf3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:31.207274 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5bce3056-ce91-4a56-b763-6a336cd1aaf3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:31.213446 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5bce3056-ce91-4a56-b763-6a336cd1aaf3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:31.213894 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5bce3056-ce91-4a56-b763-6a336cd1aaf3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:31.214650 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5bce3056-ce91-4a56-b763-6a336cd1aaf3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:31.215083 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5bce3056-ce91-4a56-b763-6a336cd1aaf3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:31.341266 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5bce3056-ce91-4a56-b763-6a336cd1aaf3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:31.348556 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5bce3056-ce91-4a56-b763-6a336cd1aaf3 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1368 microversion: 2.1 time: 0.146575 Jul 05 10:19:31.349124 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 416/804] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:31 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1368 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:32.366201 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fe21eb13-803b-4954-94dd-c291ba60b7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:32.366830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fe21eb13-803b-4954-94dd-c291ba60b7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:32.373485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fe21eb13-803b-4954-94dd-c291ba60b7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:32.373933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fe21eb13-803b-4954-94dd-c291ba60b7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:32.374780 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fe21eb13-803b-4954-94dd-c291ba60b7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:32.375241 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fe21eb13-803b-4954-94dd-c291ba60b7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:32.500089 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fe21eb13-803b-4954-94dd-c291ba60b7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:32.507451 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fe21eb13-803b-4954-94dd-c291ba60b7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1557 microversion: 2.1 time: 0.145441 Jul 05 10:19:32.508014 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 389/805] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:32 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1557 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:32.523752 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9bc07df4-f86e-445a-ba8f-6caadf407d79 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:32.524279 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9bc07df4-f86e-445a-ba8f-6caadf407d79 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:32.530555 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9bc07df4-f86e-445a-ba8f-6caadf407d79 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:32.531652 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9bc07df4-f86e-445a-ba8f-6caadf407d79 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:32.532420 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9bc07df4-f86e-445a-ba8f-6caadf407d79 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:32.532850 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9bc07df4-f86e-445a-ba8f-6caadf407d79 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:32.610213 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9bc07df4-f86e-445a-ba8f-6caadf407d79 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.090122 Jul 05 10:19:32.610814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 417/806] 10.222.0.20 () {58 vars in 1274 bytes} [Tue Jul 5 10:19:32 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14/os-interface => generated 284 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:19:33.741926 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-8b724b81-f6a3-4ca2-9ebf-8b0318b24124 req-9e632c68-efc7-49fa-953a-c9282819156c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d872dbf0-7f48-4887-85d4-9b9d0123be14", "tag": "705d8fd5-bec7-4c13-8533-3ab583c1b1bb"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:19:33.750769 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-8b724b81-f6a3-4ca2-9ebf-8b0318b24124 req-9e632c68-efc7-49fa-953a-c9282819156c service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:33.751211 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-8b724b81-f6a3-4ca2-9ebf-8b0318b24124 req-9e632c68-efc7-49fa-953a-c9282819156c service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:33.761789 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-8b724b81-f6a3-4ca2-9ebf-8b0318b24124 req-9e632c68-efc7-49fa-953a-c9282819156c service nova] Creating event network-changed:705d8fd5-bec7-4c13-8533-3ab583c1b1bb for instance d872dbf0-7f48-4887-85d4-9b9d0123be14 on hv-cinder-maste Jul 05 10:19:33.767470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-8b724b81-f6a3-4ca2-9ebf-8b0318b24124 req-9e632c68-efc7-49fa-953a-c9282819156c service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029090 Jul 05 10:19:33.769537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 390/807] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:19:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:19:35.761730 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9383efc1-38ae-4f51-8153-918ffc9190f5 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:35.762263 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9383efc1-38ae-4f51-8153-918ffc9190f5 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:35.764163 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=52,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:35.770010 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9383efc1-38ae-4f51-8153-918ffc9190f5 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:35.770459 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9383efc1-38ae-4f51-8153-918ffc9190f5 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:35.771256 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9383efc1-38ae-4f51-8153-918ffc9190f5 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:35.771712 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9383efc1-38ae-4f51-8153-918ffc9190f5 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:35.776472 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=79,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:35.784916 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-aba6a84c-b744-4ad8-9884-1e0688a49222 req-3f3b20ba-808c-4cd9-afd1-3f8410cbfd74 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4684ec3d-0407-461f-8a19-d6d8f99ca49b", "tag": "1a0fe351-4265-449d-8b7d-508a71c44b30"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:19:35.789920 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=43,nova_api:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:35.795137 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-aba6a84c-b744-4ad8-9884-1e0688a49222 req-3f3b20ba-808c-4cd9-afd1-3f8410cbfd74 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:35.795950 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-aba6a84c-b744-4ad8-9884-1e0688a49222 req-3f3b20ba-808c-4cd9-afd1-3f8410cbfd74 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:35.801944 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=79,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:35.804238 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9383efc1-38ae-4f51-8153-918ffc9190f5 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Going to try to terminate instance {{(pid=117813) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:19:35.812080 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-aba6a84c-b744-4ad8-9884-1e0688a49222 req-3f3b20ba-808c-4cd9-afd1-3f8410cbfd74 service nova] Creating event network-changed:1a0fe351-4265-449d-8b7d-508a71c44b30 for instance 4684ec3d-0407-461f-8a19-d6d8f99ca49b on hv-cinder-maste Jul 05 10:19:35.818148 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-aba6a84c-b744-4ad8-9884-1e0688a49222 req-3f3b20ba-808c-4cd9-afd1-3f8410cbfd74 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039788 Jul 05 10:19:35.819644 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 391/808] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:19:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:19:35.886715 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9383efc1-38ae-4f51-8153-918ffc9190f5 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "DELETE /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 204 len: 0 microversion: 2.1 time: 0.128835 Jul 05 10:19:35.887271 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 418/809] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:19:35 2022] DELETE /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:19:35.898292 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d31fb236-f59a-4fa6-ae29-3c09ec008969 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:35.898990 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d31fb236-f59a-4fa6-ae29-3c09ec008969 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:35.906613 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d31fb236-f59a-4fa6-ae29-3c09ec008969 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:35.907064 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d31fb236-f59a-4fa6-ae29-3c09ec008969 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:35.907843 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d31fb236-f59a-4fa6-ae29-3c09ec008969 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:35.908273 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d31fb236-f59a-4fa6-ae29-3c09ec008969 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:36.045550 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d31fb236-f59a-4fa6-ae29-3c09ec008969 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:36.052987 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d31fb236-f59a-4fa6-ae29-3c09ec008969 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.160347 Jul 05 10:19:36.053554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 392/810] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:35 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:19:37.071183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f9144ee0-e712-4583-9b65-102f6ae0d4a6 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:37.071878 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f9144ee0-e712-4583-9b65-102f6ae0d4a6 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:37.078839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f9144ee0-e712-4583-9b65-102f6ae0d4a6 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:37.079287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f9144ee0-e712-4583-9b65-102f6ae0d4a6 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:37.080056 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f9144ee0-e712-4583-9b65-102f6ae0d4a6 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:37.080521 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f9144ee0-e712-4583-9b65-102f6ae0d4a6 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:37.252038 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f9144ee0-e712-4583-9b65-102f6ae0d4a6 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:37.260082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f9144ee0-e712-4583-9b65-102f6ae0d4a6 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1568 microversion: 2.1 time: 0.192909 Jul 05 10:19:37.261030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 419/811] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:37 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1568 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:38.277947 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-dc8907bb-c1eb-42b0-9b6d-55c03fce3605 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:38.278559 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-dc8907bb-c1eb-42b0-9b6d-55c03fce3605 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:38.283048 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-849fc223-a603-492e-ba97-7e4aef29fd25 req-7bea812f-85b3-48ce-a715-f38c7ce050b0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4684ec3d-0407-461f-8a19-d6d8f99ca49b", "name": "network-vif-deleted", "tag": "1a0fe351-4265-449d-8b7d-508a71c44b30"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:19:38.286807 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dc8907bb-c1eb-42b0-9b6d-55c03fce3605 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:38.287314 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dc8907bb-c1eb-42b0-9b6d-55c03fce3605 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:38.288117 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dc8907bb-c1eb-42b0-9b6d-55c03fce3605 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:38.288710 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dc8907bb-c1eb-42b0-9b6d-55c03fce3605 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:38.294013 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-849fc223-a603-492e-ba97-7e4aef29fd25 req-7bea812f-85b3-48ce-a715-f38c7ce050b0 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:38.294449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-849fc223-a603-492e-ba97-7e4aef29fd25 req-7bea812f-85b3-48ce-a715-f38c7ce050b0 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:38.307406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-849fc223-a603-492e-ba97-7e4aef29fd25 req-7bea812f-85b3-48ce-a715-f38c7ce050b0 service nova] Creating event network-vif-deleted:1a0fe351-4265-449d-8b7d-508a71c44b30 for instance 4684ec3d-0407-461f-8a19-d6d8f99ca49b on hv-cinder-maste Jul 05 10:19:38.313006 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-849fc223-a603-492e-ba97-7e4aef29fd25 req-7bea812f-85b3-48ce-a715-f38c7ce050b0 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033956 Jul 05 10:19:38.314210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 420/812] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:19:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:19:38.353505 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-dc8907bb-c1eb-42b0-9b6d-55c03fce3605 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e322227b42b4aaa99674afd906a706a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9edab973767545988e178ddff76557ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:19:38.361741 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-dc8907bb-c1eb-42b0-9b6d-55c03fce3605 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 200 len: 1382 microversion: 2.1 time: 0.088147 Jul 05 10:19:38.362321 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 393/813] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:38 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 1382 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:19:39.379291 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a2960614-1ec5-486c-939b-1ded5728cf78 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:39.379976 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a2960614-1ec5-486c-939b-1ded5728cf78 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] [instance: 4684ec3d-0407-461f-8a19-d6d8f99ca49b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:39.386551 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a2960614-1ec5-486c-939b-1ded5728cf78 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:39.386990 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a2960614-1ec5-486c-939b-1ded5728cf78 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:39.387738 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a2960614-1ec5-486c-939b-1ded5728cf78 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:39.388195 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a2960614-1ec5-486c-939b-1ded5728cf78 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:39.401527 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-a2960614-1ec5-486c-939b-1ded5728cf78 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] HTTP exception thrown: Instance 4684ec3d-0407-461f-8a19-d6d8f99ca49b could not be found. Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a2960614-1ec5-486c-939b-1ded5728cf78 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Request method failure captured: Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b HTTP/1.1 Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA6UP2BuCsPIOwoYeX-SJ-XvDIJn_XIaJ5h1feHgtXQZmei8Ib3d3Q-P8XFWxoQRuH_1fnPSfuMUqHQAXtgeEtvQmlYGHmVnLskGcZrtaI1BifHyXoY8QikvwYihX9lLhg0n3T7MZngB-sTX9JhFMANhMGj63UGoG-6UHjOJje0L0WI Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 9edab973767545988e178ddff76557ac Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-TestShelveInstance-801082525 Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:19:39.403089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:19:39.404700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/9edab973767545988e178ddff76557ac"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_9edab973767545988e178ddff76557ac"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9edab973767545988e178ddff76557ac"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/9edab973767545988e178ddff76557ac"}]}] Jul 05 10:19:39.404700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-TestShelveInstance-801082525 Jul 05 10:19:39.404700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 9edab973767545988e178ddff76557ac Jul 05 10:19:39.404700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-TestShelveInstance-801082525 Jul 05 10:19:39.404700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-TestShelveInstance-801082525-project Jul 05 10:19:39.404700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:19:39.404700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:19:39.404700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 7e322227b42b4aaa99674afd906a706a Jul 05 10:19:39.404700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-TestShelveInstance-801082525-project Jul 05 10:19:39.404700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:19:39.404700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance 4684ec3d-0407-461f-8a19-d6d8f99ca49b could not be found. Jul 05 10:19:39.404700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': '4684ec3d-0407-461f-8a19-d6d8f99ca49b'} Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4684ec3d-0407-461f-8a19-d6d8f99ca49b could not be found. Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:19:39.405678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4684ec3d-0407-461f-8a19-d6d8f99ca49b could not be found. Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4684ec3d-0407-461f-8a19-d6d8f99ca49b could not be found. Jul 05 10:19:39.407748 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:19:39.409192 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a2960614-1ec5-486c-939b-1ded5728cf78 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Returning 404 to user: Instance 4684ec3d-0407-461f-8a19-d6d8f99ca49b could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:19:39.409192 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a2960614-1ec5-486c-939b-1ded5728cf78 tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b" status: 404 len: 111 microversion: 2.1 time: 0.029367 Jul 05 10:19:39.409192 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 421/814] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:19:39 2022] GET /compute/v2.1/servers/4684ec3d-0407-461f-8a19-d6d8f99ca49b => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:19:39.565877 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-594ac843-80f1-4027-b5ab-4bb07003b65a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] Calling method '.version_select at 0x7feb89586f70>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:39.585549 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-594ac843-80f1-4027-b5ab-4bb07003b65a tempest-TestShelveInstance-801082525 tempest-TestShelveInstance-801082525-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-309513784" status: 202 len: 0 microversion: 2.1 time: 0.023903 Jul 05 10:19:39.586172 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 394/815] 10.222.0.20 () {58 vars in 1250 bytes} [Tue Jul 5 10:19:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-309513784 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:19:59.811409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-37b33c21-f14f-48f7-a7b7-804884969c37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:19:59.811984 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-37b33c21-f14f-48f7-a7b7-804884969c37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:19:59.814699 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=12,nova_api:UPDATE=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:59.815354 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=23,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:19:59.821476 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-37b33c21-f14f-48f7-a7b7-804884969c37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:59.821917 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-37b33c21-f14f-48f7-a7b7-804884969c37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:59.822673 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-37b33c21-f14f-48f7-a7b7-804884969c37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:19:59.823101 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-37b33c21-f14f-48f7-a7b7-804884969c37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:19:59.849163 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-37b33c21-f14f-48f7-a7b7-804884969c37 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/ips" status: 200 len: 151 microversion: 2.20 time: 0.041703 Jul 05 10:19:59.849725 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 422/816] 10.222.0.20 () {60 vars in 1288 bytes} [Tue Jul 5 10:19:59 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/ips => generated 151 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:20:01.639000 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=13 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:20:01.639687 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=8,nova_api:DELETE=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:20:01.644277 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-47c226f3-015e-436f-b3a5-d7ba0910c735 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:20:01.644864 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-47c226f3-015e-436f-b3a5-d7ba0910c735 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:01.654836 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-47c226f3-015e-436f-b3a5-d7ba0910c735 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:01.655285 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-47c226f3-015e-436f-b3a5-d7ba0910c735 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:01.656055 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-47c226f3-015e-436f-b3a5-d7ba0910c735 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:01.656520 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-47c226f3-015e-436f-b3a5-d7ba0910c735 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:01.679164 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-47c226f3-015e-436f-b3a5-d7ba0910c735 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lazy-loading 'flavor' on Instance uuid ff442846-3ded-48c4-8a48-df125f46d0d6 {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:20:01.935283 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-47c226f3-015e-436f-b3a5-d7ba0910c735 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "POST /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/action" status: 202 len: 0 microversion: 2.20 time: 0.297583 Jul 05 10:20:01.936036 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 395/817] 10.222.0.20 () {62 vars in 1318 bytes} [Tue Jul 5 10:20:01 2022] POST /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/action => generated 0 bytes in 299 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:20:01.944567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0d665c4b-6564-478a-9e29-f160c9d011de tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:01.945200 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0d665c4b-6564-478a-9e29-f160c9d011de tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:01.951648 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0d665c4b-6564-478a-9e29-f160c9d011de tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:01.952127 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0d665c4b-6564-478a-9e29-f160c9d011de tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:01.952865 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0d665c4b-6564-478a-9e29-f160c9d011de tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:01.953302 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0d665c4b-6564-478a-9e29-f160c9d011de tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:02.082602 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0d665c4b-6564-478a-9e29-f160c9d011de tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:02.090382 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0d665c4b-6564-478a-9e29-f160c9d011de tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:02.091863 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0d665c4b-6564-478a-9e29-f160c9d011de tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:02.093035 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0d665c4b-6564-478a-9e29-f160c9d011de tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1723 microversion: 2.20 time: 0.152293 Jul 05 10:20:02.093587 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 423/818] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:01 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:03.112234 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-4c224796-76b8-479f-a1b4-cd589cb8e467 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:03.112953 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-4c224796-76b8-479f-a1b4-cd589cb8e467 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:03.119413 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4c224796-76b8-479f-a1b4-cd589cb8e467 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:03.119865 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4c224796-76b8-479f-a1b4-cd589cb8e467 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:03.120655 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4c224796-76b8-479f-a1b4-cd589cb8e467 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:03.121094 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4c224796-76b8-479f-a1b4-cd589cb8e467 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:03.253140 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-4c224796-76b8-479f-a1b4-cd589cb8e467 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:03.261032 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-4c224796-76b8-479f-a1b4-cd589cb8e467 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:03.262529 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-4c224796-76b8-479f-a1b4-cd589cb8e467 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:03.263672 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-4c224796-76b8-479f-a1b4-cd589cb8e467 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1723 microversion: 2.20 time: 0.156227 Jul 05 10:20:03.264237 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 396/819] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:03 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1723 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:04.281625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8071d501-2447-4c79-b4c7-9f5cb93ec5f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:04.282226 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8071d501-2447-4c79-b4c7-9f5cb93ec5f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:04.289420 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8071d501-2447-4c79-b4c7-9f5cb93ec5f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:04.289865 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8071d501-2447-4c79-b4c7-9f5cb93ec5f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:04.290620 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8071d501-2447-4c79-b4c7-9f5cb93ec5f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:04.291057 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8071d501-2447-4c79-b4c7-9f5cb93ec5f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:04.416305 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8071d501-2447-4c79-b4c7-9f5cb93ec5f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:04.424210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8071d501-2447-4c79-b4c7-9f5cb93ec5f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:04.425720 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8071d501-2447-4c79-b4c7-9f5cb93ec5f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:04.426853 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8071d501-2447-4c79-b4c7-9f5cb93ec5f2 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1723 microversion: 2.20 time: 0.149133 Jul 05 10:20:04.427403 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 424/820] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:04 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1723 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:05.445509 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-76ce4f4b-2976-4540-bece-04280b3c3e74 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:05.446126 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-76ce4f4b-2976-4540-bece-04280b3c3e74 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:05.452850 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-76ce4f4b-2976-4540-bece-04280b3c3e74 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:05.453306 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-76ce4f4b-2976-4540-bece-04280b3c3e74 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:05.454074 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-76ce4f4b-2976-4540-bece-04280b3c3e74 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:05.454518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-76ce4f4b-2976-4540-bece-04280b3c3e74 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:05.581959 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-76ce4f4b-2976-4540-bece-04280b3c3e74 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:05.589761 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-76ce4f4b-2976-4540-bece-04280b3c3e74 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:05.591273 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-76ce4f4b-2976-4540-bece-04280b3c3e74 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:05.592454 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-76ce4f4b-2976-4540-bece-04280b3c3e74 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1723 microversion: 2.20 time: 0.150773 Jul 05 10:20:05.593021 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 397/821] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:05 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1723 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:06.610551 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-12c0a060-a0cd-4c7b-b592-b78dec063345 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:06.611184 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-12c0a060-a0cd-4c7b-b592-b78dec063345 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:06.617899 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-12c0a060-a0cd-4c7b-b592-b78dec063345 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:06.618347 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-12c0a060-a0cd-4c7b-b592-b78dec063345 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:06.619102 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-12c0a060-a0cd-4c7b-b592-b78dec063345 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:06.619535 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-12c0a060-a0cd-4c7b-b592-b78dec063345 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:06.945147 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-12c0a060-a0cd-4c7b-b592-b78dec063345 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:06.952881 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-12c0a060-a0cd-4c7b-b592-b78dec063345 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:06.954363 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-12c0a060-a0cd-4c7b-b592-b78dec063345 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:06.955518 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-12c0a060-a0cd-4c7b-b592-b78dec063345 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1723 microversion: 2.20 time: 0.348902 Jul 05 10:20:06.956073 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 425/822] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:06 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1723 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:07.973370 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6ecf5a64-dd04-4c45-a663-de3843e8c192 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:07.973995 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6ecf5a64-dd04-4c45-a663-de3843e8c192 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:07.980385 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6ecf5a64-dd04-4c45-a663-de3843e8c192 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:07.980887 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6ecf5a64-dd04-4c45-a663-de3843e8c192 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:07.981654 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6ecf5a64-dd04-4c45-a663-de3843e8c192 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:07.982091 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6ecf5a64-dd04-4c45-a663-de3843e8c192 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:08.163454 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6ecf5a64-dd04-4c45-a663-de3843e8c192 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:08.171181 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6ecf5a64-dd04-4c45-a663-de3843e8c192 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:08.172713 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6ecf5a64-dd04-4c45-a663-de3843e8c192 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:08.174468 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6ecf5a64-dd04-4c45-a663-de3843e8c192 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1723 microversion: 2.20 time: 0.204399 Jul 05 10:20:08.175025 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 398/823] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:07 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1723 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:09.193042 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-4d18c7c6-e583-45f6-bc61-78dec7263fce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:09.193646 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-4d18c7c6-e583-45f6-bc61-78dec7263fce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:09.199971 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4d18c7c6-e583-45f6-bc61-78dec7263fce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:09.200434 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4d18c7c6-e583-45f6-bc61-78dec7263fce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:09.201187 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4d18c7c6-e583-45f6-bc61-78dec7263fce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:09.201617 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4d18c7c6-e583-45f6-bc61-78dec7263fce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:09.329624 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4d18c7c6-e583-45f6-bc61-78dec7263fce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:09.337342 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4d18c7c6-e583-45f6-bc61-78dec7263fce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:09.338835 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4d18c7c6-e583-45f6-bc61-78dec7263fce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:09.339971 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-4d18c7c6-e583-45f6-bc61-78dec7263fce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1723 microversion: 2.20 time: 0.151439 Jul 05 10:20:09.340554 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 426/824] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:09 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:10.358380 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fee46189-0dc9-479e-afdc-b0a2c1ead125 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:10.358984 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fee46189-0dc9-479e-afdc-b0a2c1ead125 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:10.365575 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fee46189-0dc9-479e-afdc-b0a2c1ead125 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:10.366021 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fee46189-0dc9-479e-afdc-b0a2c1ead125 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:10.366779 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fee46189-0dc9-479e-afdc-b0a2c1ead125 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:10.367211 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fee46189-0dc9-479e-afdc-b0a2c1ead125 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:10.491428 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fee46189-0dc9-479e-afdc-b0a2c1ead125 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:10.499091 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fee46189-0dc9-479e-afdc-b0a2c1ead125 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:10.500793 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fee46189-0dc9-479e-afdc-b0a2c1ead125 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:10.501737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fee46189-0dc9-479e-afdc-b0a2c1ead125 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1723 microversion: 2.20 time: 0.147255 Jul 05 10:20:10.502293 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 399/825] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:10 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1723 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:11.520720 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e92e253e-9d7f-40ad-9ba2-90d80c96ce07 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:11.521319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e92e253e-9d7f-40ad-9ba2-90d80c96ce07 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:11.527702 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e92e253e-9d7f-40ad-9ba2-90d80c96ce07 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:11.528147 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e92e253e-9d7f-40ad-9ba2-90d80c96ce07 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:11.528927 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e92e253e-9d7f-40ad-9ba2-90d80c96ce07 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:11.529357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e92e253e-9d7f-40ad-9ba2-90d80c96ce07 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:11.656739 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e92e253e-9d7f-40ad-9ba2-90d80c96ce07 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:11.665157 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e92e253e-9d7f-40ad-9ba2-90d80c96ce07 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:11.666644 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e92e253e-9d7f-40ad-9ba2-90d80c96ce07 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:11.667776 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e92e253e-9d7f-40ad-9ba2-90d80c96ce07 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1723 microversion: 2.20 time: 0.151485 Jul 05 10:20:11.668332 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 427/826] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:11 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:12.685828 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2cade1ef-e4c5-4eaf-b23c-068dbb5dc2b0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:12.686433 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2cade1ef-e4c5-4eaf-b23c-068dbb5dc2b0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:12.693116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2cade1ef-e4c5-4eaf-b23c-068dbb5dc2b0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:12.693566 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2cade1ef-e4c5-4eaf-b23c-068dbb5dc2b0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:12.694326 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2cade1ef-e4c5-4eaf-b23c-068dbb5dc2b0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:12.694763 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2cade1ef-e4c5-4eaf-b23c-068dbb5dc2b0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:12.823976 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-2cade1ef-e4c5-4eaf-b23c-068dbb5dc2b0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:12.831778 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-2cade1ef-e4c5-4eaf-b23c-068dbb5dc2b0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:12.833316 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-2cade1ef-e4c5-4eaf-b23c-068dbb5dc2b0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:12.834455 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2cade1ef-e4c5-4eaf-b23c-068dbb5dc2b0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1723 microversion: 2.20 time: 0.152530 Jul 05 10:20:12.835017 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 400/827] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:12 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:13.852510 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0ae1f627-3d13-4195-b2cf-aaae91f73b51 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:13.853145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0ae1f627-3d13-4195-b2cf-aaae91f73b51 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:13.859722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0ae1f627-3d13-4195-b2cf-aaae91f73b51 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:13.860200 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0ae1f627-3d13-4195-b2cf-aaae91f73b51 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:13.860988 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0ae1f627-3d13-4195-b2cf-aaae91f73b51 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:13.861425 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0ae1f627-3d13-4195-b2cf-aaae91f73b51 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:13.990198 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0ae1f627-3d13-4195-b2cf-aaae91f73b51 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:13.998355 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0ae1f627-3d13-4195-b2cf-aaae91f73b51 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:13.999863 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0ae1f627-3d13-4195-b2cf-aaae91f73b51 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:14.001058 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0ae1f627-3d13-4195-b2cf-aaae91f73b51 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1744 microversion: 2.20 time: 0.152443 Jul 05 10:20:14.001820 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 428/828] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:13 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1744 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:15.018978 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-264cc5d4-c1b6-4b1b-b67b-064a38b9b531 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:15.019593 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-264cc5d4-c1b6-4b1b-b67b-064a38b9b531 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:15.026022 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-264cc5d4-c1b6-4b1b-b67b-064a38b9b531 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:15.026488 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-264cc5d4-c1b6-4b1b-b67b-064a38b9b531 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:15.027237 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-264cc5d4-c1b6-4b1b-b67b-064a38b9b531 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:15.027676 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-264cc5d4-c1b6-4b1b-b67b-064a38b9b531 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:15.157916 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-264cc5d4-c1b6-4b1b-b67b-064a38b9b531 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:15.166498 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-264cc5d4-c1b6-4b1b-b67b-064a38b9b531 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:15.167986 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-264cc5d4-c1b6-4b1b-b67b-064a38b9b531 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:15.169175 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-264cc5d4-c1b6-4b1b-b67b-064a38b9b531 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1744 microversion: 2.20 time: 0.154097 Jul 05 10:20:15.169733 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 401/829] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:15 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1744 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:16.187269 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d7a57006-367b-47a9-9962-63d8b2844469 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:16.187799 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d7a57006-367b-47a9-9962-63d8b2844469 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:16.194397 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d7a57006-367b-47a9-9962-63d8b2844469 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:16.194838 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d7a57006-367b-47a9-9962-63d8b2844469 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:16.195591 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d7a57006-367b-47a9-9962-63d8b2844469 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:16.196016 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d7a57006-367b-47a9-9962-63d8b2844469 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:16.327116 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d7a57006-367b-47a9-9962-63d8b2844469 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:16.335984 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d7a57006-367b-47a9-9962-63d8b2844469 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:16.337523 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d7a57006-367b-47a9-9962-63d8b2844469 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:16.338846 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d7a57006-367b-47a9-9962-63d8b2844469 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1744 microversion: 2.20 time: 0.155502 Jul 05 10:20:16.339755 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 429/830] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:16 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1744 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:17.356998 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b352db07-2cdf-4057-af75-189c02db7278 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:17.357711 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b352db07-2cdf-4057-af75-189c02db7278 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:17.364164 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b352db07-2cdf-4057-af75-189c02db7278 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:17.364646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b352db07-2cdf-4057-af75-189c02db7278 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:17.365441 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b352db07-2cdf-4057-af75-189c02db7278 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:17.366150 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b352db07-2cdf-4057-af75-189c02db7278 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:17.498472 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b352db07-2cdf-4057-af75-189c02db7278 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:17.506930 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b352db07-2cdf-4057-af75-189c02db7278 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:17.508446 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b352db07-2cdf-4057-af75-189c02db7278 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:17.509593 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b352db07-2cdf-4057-af75-189c02db7278 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1744 microversion: 2.20 time: 0.156443 Jul 05 10:20:17.510147 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 402/831] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:17 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1744 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:18.400759 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-3474fd83-06b6-40e5-bffe-ea9176c95519 req-4affd025-d20e-41a1-8b09-81f5e3cc472b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ff442846-3ded-48c4-8a48-df125f46d0d6", "name": "network-vif-plugged", "status": "completed", "tag": "e6fb0593-584d-4308-86f5-4705f3dc20c8"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:20:18.410834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-3474fd83-06b6-40e5-bffe-ea9176c95519 req-4affd025-d20e-41a1-8b09-81f5e3cc472b service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:18.411281 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-3474fd83-06b6-40e5-bffe-ea9176c95519 req-4affd025-d20e-41a1-8b09-81f5e3cc472b service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:18.422712 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-3474fd83-06b6-40e5-bffe-ea9176c95519 req-4affd025-d20e-41a1-8b09-81f5e3cc472b service nova] Creating event network-vif-plugged:e6fb0593-584d-4308-86f5-4705f3dc20c8 for instance ff442846-3ded-48c4-8a48-df125f46d0d6 on hv-cinder-maste Jul 05 10:20:18.428027 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-3474fd83-06b6-40e5-bffe-ea9176c95519 req-4affd025-d20e-41a1-8b09-81f5e3cc472b service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030776 Jul 05 10:20:18.428804 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 430/832] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:20:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:20:18.527519 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-9e4ee3b0-3acc-4949-8e20-f86efb189db4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:18.528056 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-9e4ee3b0-3acc-4949-8e20-f86efb189db4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:18.534498 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9e4ee3b0-3acc-4949-8e20-f86efb189db4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:18.534945 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9e4ee3b0-3acc-4949-8e20-f86efb189db4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:18.535701 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9e4ee3b0-3acc-4949-8e20-f86efb189db4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:18.536134 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-9e4ee3b0-3acc-4949-8e20-f86efb189db4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:18.663456 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-9e4ee3b0-3acc-4949-8e20-f86efb189db4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:18.671204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-9e4ee3b0-3acc-4949-8e20-f86efb189db4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:18.672713 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-9e4ee3b0-3acc-4949-8e20-f86efb189db4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:18.673843 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-9e4ee3b0-3acc-4949-8e20-f86efb189db4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1744 microversion: 2.20 time: 0.150339 Jul 05 10:20:18.674394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 403/833] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:18 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1744 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:19.692190 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-89db6ee6-7112-4712-91e5-fa4f9b3f6043 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:19.692916 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-89db6ee6-7112-4712-91e5-fa4f9b3f6043 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:19.699946 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-89db6ee6-7112-4712-91e5-fa4f9b3f6043 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:19.700218 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-89db6ee6-7112-4712-91e5-fa4f9b3f6043 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:19.701079 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-89db6ee6-7112-4712-91e5-fa4f9b3f6043 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:19.701444 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-89db6ee6-7112-4712-91e5-fa4f9b3f6043 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:19.851689 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-89db6ee6-7112-4712-91e5-fa4f9b3f6043 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:19.861052 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-89db6ee6-7112-4712-91e5-fa4f9b3f6043 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:19.864638 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-89db6ee6-7112-4712-91e5-fa4f9b3f6043 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:19.865815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-89db6ee6-7112-4712-91e5-fa4f9b3f6043 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1739 microversion: 2.20 time: 0.178604 Jul 05 10:20:19.867037 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 431/834] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:19 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1739 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:20.885136 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d74f9513-4c62-4af9-911e-8095ebfc2f9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:20.886551 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d74f9513-4c62-4af9-911e-8095ebfc2f9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:20.893949 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d74f9513-4c62-4af9-911e-8095ebfc2f9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:20.894638 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d74f9513-4c62-4af9-911e-8095ebfc2f9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:20.895684 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d74f9513-4c62-4af9-911e-8095ebfc2f9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:20.896307 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d74f9513-4c62-4af9-911e-8095ebfc2f9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:21.355048 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d74f9513-4c62-4af9-911e-8095ebfc2f9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:21.363487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d74f9513-4c62-4af9-911e-8095ebfc2f9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:21.365106 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d74f9513-4c62-4af9-911e-8095ebfc2f9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:21.366552 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d74f9513-4c62-4af9-911e-8095ebfc2f9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1739 microversion: 2.20 time: 0.485027 Jul 05 10:20:21.377330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 404/835] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:20 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1739 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:22.384783 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6c1eaffd-da38-4ab9-b0c4-08378fe01af6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:22.386105 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-6c1eaffd-da38-4ab9-b0c4-08378fe01af6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:22.393112 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6c1eaffd-da38-4ab9-b0c4-08378fe01af6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:22.393562 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6c1eaffd-da38-4ab9-b0c4-08378fe01af6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:22.394507 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6c1eaffd-da38-4ab9-b0c4-08378fe01af6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:22.394980 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6c1eaffd-da38-4ab9-b0c4-08378fe01af6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:22.538708 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6c1eaffd-da38-4ab9-b0c4-08378fe01af6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:22.547339 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6c1eaffd-da38-4ab9-b0c4-08378fe01af6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:22.549040 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6c1eaffd-da38-4ab9-b0c4-08378fe01af6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:22.550037 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-6c1eaffd-da38-4ab9-b0c4-08378fe01af6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1739 microversion: 2.20 time: 0.169161 Jul 05 10:20:22.551219 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 432/836] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:22 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1739 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 05 10:20:23.569532 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ce070435-2dfc-47c6-8af9-a4d63c6159ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:23.570122 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ce070435-2dfc-47c6-8af9-a4d63c6159ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:23.576776 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ce070435-2dfc-47c6-8af9-a4d63c6159ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:23.577231 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ce070435-2dfc-47c6-8af9-a4d63c6159ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:23.577998 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ce070435-2dfc-47c6-8af9-a4d63c6159ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:23.578438 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ce070435-2dfc-47c6-8af9-a4d63c6159ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:23.712176 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ce070435-2dfc-47c6-8af9-a4d63c6159ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:23.720160 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ce070435-2dfc-47c6-8af9-a4d63c6159ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:23.721765 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ce070435-2dfc-47c6-8af9-a4d63c6159ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:23.723554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ce070435-2dfc-47c6-8af9-a4d63c6159ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1739 microversion: 2.20 time: 0.157901 Jul 05 10:20:23.724353 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 405/837] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:23 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1739 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:24.741915 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9dbf66f5-91a2-477c-b6be-d1a8fbecd4fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:24.742727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9dbf66f5-91a2-477c-b6be-d1a8fbecd4fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:24.749219 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9dbf66f5-91a2-477c-b6be-d1a8fbecd4fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:24.749664 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9dbf66f5-91a2-477c-b6be-d1a8fbecd4fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:24.750425 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9dbf66f5-91a2-477c-b6be-d1a8fbecd4fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:24.750860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9dbf66f5-91a2-477c-b6be-d1a8fbecd4fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:24.893696 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9dbf66f5-91a2-477c-b6be-d1a8fbecd4fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:24.902604 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9dbf66f5-91a2-477c-b6be-d1a8fbecd4fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:24.904149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9dbf66f5-91a2-477c-b6be-d1a8fbecd4fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:24.905895 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9dbf66f5-91a2-477c-b6be-d1a8fbecd4fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1739 microversion: 2.20 time: 0.167647 Jul 05 10:20:24.907308 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 433/838] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:24 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1739 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:25.934788 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-58512fff-735f-42a4-8934-bfc3c8835c5b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:25.935429 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-58512fff-735f-42a4-8934-bfc3c8835c5b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:25.943158 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-58512fff-735f-42a4-8934-bfc3c8835c5b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:25.943855 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-58512fff-735f-42a4-8934-bfc3c8835c5b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:25.945725 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-58512fff-735f-42a4-8934-bfc3c8835c5b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:25.946228 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-58512fff-735f-42a4-8934-bfc3c8835c5b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:26.101696 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-58512fff-735f-42a4-8934-bfc3c8835c5b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:26.112688 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-58512fff-735f-42a4-8934-bfc3c8835c5b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:26.114459 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-58512fff-735f-42a4-8934-bfc3c8835c5b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:26.115807 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-58512fff-735f-42a4-8934-bfc3c8835c5b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1739 microversion: 2.20 time: 0.187108 Jul 05 10:20:26.116984 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 406/839] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:25 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1739 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:27.135430 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-01dae2cd-0784-4871-9f90-2683f6b48d99 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:27.136694 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-01dae2cd-0784-4871-9f90-2683f6b48d99 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:27.142889 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-01dae2cd-0784-4871-9f90-2683f6b48d99 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:27.143566 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-01dae2cd-0784-4871-9f90-2683f6b48d99 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:27.144343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-01dae2cd-0784-4871-9f90-2683f6b48d99 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:27.144807 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-01dae2cd-0784-4871-9f90-2683f6b48d99 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:27.298490 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-01dae2cd-0784-4871-9f90-2683f6b48d99 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:27.311027 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-01dae2cd-0784-4871-9f90-2683f6b48d99 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:27.312564 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-01dae2cd-0784-4871-9f90-2683f6b48d99 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:27.314688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-01dae2cd-0784-4871-9f90-2683f6b48d99 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1739 microversion: 2.20 time: 0.184015 Jul 05 10:20:27.316179 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 434/840] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:27 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1739 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:28.343325 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-bebbd9ae-7d32-435d-a541-0af40b06cbba tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:28.343926 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bebbd9ae-7d32-435d-a541-0af40b06cbba tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:28.351806 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bebbd9ae-7d32-435d-a541-0af40b06cbba tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:28.352206 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bebbd9ae-7d32-435d-a541-0af40b06cbba tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:28.353027 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bebbd9ae-7d32-435d-a541-0af40b06cbba tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:28.353477 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bebbd9ae-7d32-435d-a541-0af40b06cbba tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:28.507783 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-bebbd9ae-7d32-435d-a541-0af40b06cbba tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:28.516026 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-bebbd9ae-7d32-435d-a541-0af40b06cbba tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:28.517705 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-bebbd9ae-7d32-435d-a541-0af40b06cbba tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:28.518992 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-bebbd9ae-7d32-435d-a541-0af40b06cbba tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1739 microversion: 2.20 time: 0.179719 Jul 05 10:20:28.519610 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 407/841] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:28 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1739 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:29.536993 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9f11e724-56fe-4bac-be09-25f597c69230 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:29.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9f11e724-56fe-4bac-be09-25f597c69230 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:29.544213 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9f11e724-56fe-4bac-be09-25f597c69230 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:29.544789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9f11e724-56fe-4bac-be09-25f597c69230 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:29.545554 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9f11e724-56fe-4bac-be09-25f597c69230 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:29.545988 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9f11e724-56fe-4bac-be09-25f597c69230 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:29.672638 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9f11e724-56fe-4bac-be09-25f597c69230 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:29.681567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9f11e724-56fe-4bac-be09-25f597c69230 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:29.683065 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-9f11e724-56fe-4bac-be09-25f597c69230 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:29.684207 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9f11e724-56fe-4bac-be09-25f597c69230 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1739 microversion: 2.20 time: 0.151014 Jul 05 10:20:29.684805 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 435/842] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:29 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1739 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:30.702342 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a57c1bcb-ba88-4c95-9a2d-e7e6620c9a0d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:30.703071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a57c1bcb-ba88-4c95-9a2d-e7e6620c9a0d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:30.709390 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a57c1bcb-ba88-4c95-9a2d-e7e6620c9a0d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:30.709848 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a57c1bcb-ba88-4c95-9a2d-e7e6620c9a0d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:30.710614 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a57c1bcb-ba88-4c95-9a2d-e7e6620c9a0d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:30.711060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a57c1bcb-ba88-4c95-9a2d-e7e6620c9a0d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:30.844556 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a57c1bcb-ba88-4c95-9a2d-e7e6620c9a0d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:30.853034 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a57c1bcb-ba88-4c95-9a2d-e7e6620c9a0d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:30.854536 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a57c1bcb-ba88-4c95-9a2d-e7e6620c9a0d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:30.855688 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a57c1bcb-ba88-4c95-9a2d-e7e6620c9a0d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1721 microversion: 2.20 time: 0.157304 Jul 05 10:20:30.856251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 408/843] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:30 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1721 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:31.874866 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8c18a127-271d-4899-b350-48b9d534b2a4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:31.875470 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8c18a127-271d-4899-b350-48b9d534b2a4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:31.877942 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=29 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:20:31.883982 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8c18a127-271d-4899-b350-48b9d534b2a4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:31.884456 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8c18a127-271d-4899-b350-48b9d534b2a4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:31.885218 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8c18a127-271d-4899-b350-48b9d534b2a4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:31.885651 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8c18a127-271d-4899-b350-48b9d534b2a4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:31.889670 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=79 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:20:32.014848 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8c18a127-271d-4899-b350-48b9d534b2a4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:32.022777 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8c18a127-271d-4899-b350-48b9d534b2a4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:32.024930 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8c18a127-271d-4899-b350-48b9d534b2a4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:32.026084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8c18a127-271d-4899-b350-48b9d534b2a4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1721 microversion: 2.20 time: 0.155072 Jul 05 10:20:32.026647 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 436/844] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:31 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1721 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:33.044552 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5c886405-ecdf-421f-86a7-cdac87287b67 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:33.045186 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5c886405-ecdf-421f-86a7-cdac87287b67 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:33.048562 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=29 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:20:33.052848 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5c886405-ecdf-421f-86a7-cdac87287b67 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:33.053254 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5c886405-ecdf-421f-86a7-cdac87287b67 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:33.054098 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5c886405-ecdf-421f-86a7-cdac87287b67 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:33.055085 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5c886405-ecdf-421f-86a7-cdac87287b67 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:33.057728 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:20:33.184996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5c886405-ecdf-421f-86a7-cdac87287b67 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:33.192739 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5c886405-ecdf-421f-86a7-cdac87287b67 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:33.194233 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5c886405-ecdf-421f-86a7-cdac87287b67 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:33.195365 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5c886405-ecdf-421f-86a7-cdac87287b67 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1664 microversion: 2.20 time: 0.154869 Jul 05 10:20:33.195920 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 409/845] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:33 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1664 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:33.213090 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-afd0f7c3-073f-4503-ad73-0ec9dcc353da tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b0817488-9309-47d3-8022-3cfc6627eefb"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:20:33.214139 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-afd0f7c3-073f-4503-ad73-0ec9dcc353da tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:33.220640 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-afd0f7c3-073f-4503-ad73-0ec9dcc353da tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:33.221083 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-afd0f7c3-073f-4503-ad73-0ec9dcc353da tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:33.221832 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-afd0f7c3-073f-4503-ad73-0ec9dcc353da tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:33.222256 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-afd0f7c3-073f-4503-ad73-0ec9dcc353da tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:33.487167 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-afd0f7c3-073f-4503-ad73-0ec9dcc353da tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "POST /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.278503 Jul 05 10:20:33.487776 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 437/846] 10.222.0.20 () {62 vars in 1363 bytes} [Tue Jul 5 10:20:33 2022] POST /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments => generated 188 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:20:33.599529 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6d222616-07c3-4cfa-b135-e3ce3cc7f48a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:20:33.600098 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6d222616-07c3-4cfa-b135-e3ce3cc7f48a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:33.606676 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6d222616-07c3-4cfa-b135-e3ce3cc7f48a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:33.607144 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6d222616-07c3-4cfa-b135-e3ce3cc7f48a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:33.607900 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6d222616-07c3-4cfa-b135-e3ce3cc7f48a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:33.608345 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6d222616-07c3-4cfa-b135-e3ce3cc7f48a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:33.681997 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6d222616-07c3-4cfa-b135-e3ce3cc7f48a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "POST /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/action" status: 202 len: 0 microversion: 2.20 time: 0.086365 Jul 05 10:20:33.682595 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 410/847] 10.222.0.20 () {62 vars in 1318 bytes} [Tue Jul 5 10:20:33 2022] POST /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/action => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:20:33.690739 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f328cea5-b25d-4ad2-9376-0174ff4edd83 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:33.691349 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f328cea5-b25d-4ad2-9376-0174ff4edd83 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:33.698582 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f328cea5-b25d-4ad2-9376-0174ff4edd83 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:33.699033 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f328cea5-b25d-4ad2-9376-0174ff4edd83 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:33.699803 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f328cea5-b25d-4ad2-9376-0174ff4edd83 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:33.700248 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f328cea5-b25d-4ad2-9376-0174ff4edd83 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:33.838273 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f328cea5-b25d-4ad2-9376-0174ff4edd83 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:33.846586 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f328cea5-b25d-4ad2-9376-0174ff4edd83 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:33.848083 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f328cea5-b25d-4ad2-9376-0174ff4edd83 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:33.849261 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f328cea5-b25d-4ad2-9376-0174ff4edd83 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1750 microversion: 2.20 time: 0.162452 Jul 05 10:20:33.849819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 438/848] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:33 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1750 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 05 10:20:34.493508 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-b923fb45-c7a9-40d4-9ad2-376678b509a0 req-9f30cb66-7587-4d78-9298-810c09225012 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ff442846-3ded-48c4-8a48-df125f46d0d6", "name": "network-vif-unplugged", "status": "completed", "tag": "e6fb0593-584d-4308-86f5-4705f3dc20c8"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:20:34.502437 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-b923fb45-c7a9-40d4-9ad2-376678b509a0 req-9f30cb66-7587-4d78-9298-810c09225012 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:34.502989 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-b923fb45-c7a9-40d4-9ad2-376678b509a0 req-9f30cb66-7587-4d78-9298-810c09225012 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:34.515229 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.compute.server_external_events [req-b923fb45-c7a9-40d4-9ad2-376678b509a0 req-9f30cb66-7587-4d78-9298-810c09225012 service nova] Unable to find a host for instance ff442846-3ded-48c4-8a48-df125f46d0d6. Dropping event network-vif-unplugged {{(pid=117812) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Jul 05 10:20:34.516452 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-b923fb45-c7a9-40d4-9ad2-376678b509a0 req-9f30cb66-7587-4d78-9298-810c09225012 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.027494 Jul 05 10:20:34.516993 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 411/849] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:20:34 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 29 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Jul 05 10:20:34.868764 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8ccd6a7c-fbfe-4cf5-8fae-d79ec0230d7f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:34.869374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8ccd6a7c-fbfe-4cf5-8fae-d79ec0230d7f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:34.875965 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ccd6a7c-fbfe-4cf5-8fae-d79ec0230d7f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:34.876433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ccd6a7c-fbfe-4cf5-8fae-d79ec0230d7f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:34.877203 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ccd6a7c-fbfe-4cf5-8fae-d79ec0230d7f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:34.877631 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8ccd6a7c-fbfe-4cf5-8fae-d79ec0230d7f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:35.004127 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8ccd6a7c-fbfe-4cf5-8fae-d79ec0230d7f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:35.012898 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8ccd6a7c-fbfe-4cf5-8fae-d79ec0230d7f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:35.014398 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8ccd6a7c-fbfe-4cf5-8fae-d79ec0230d7f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:35.015550 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8ccd6a7c-fbfe-4cf5-8fae-d79ec0230d7f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1752 microversion: 2.20 time: 0.151104 Jul 05 10:20:35.016120 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 439/850] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:34 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1752 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:36.035015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-53fd5d10-1fcd-4026-a11b-f1476035289f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:36.035643 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-53fd5d10-1fcd-4026-a11b-f1476035289f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:36.042826 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-53fd5d10-1fcd-4026-a11b-f1476035289f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:36.043278 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-53fd5d10-1fcd-4026-a11b-f1476035289f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:36.044034 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-53fd5d10-1fcd-4026-a11b-f1476035289f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:36.044499 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-53fd5d10-1fcd-4026-a11b-f1476035289f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:36.244876 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-53fd5d10-1fcd-4026-a11b-f1476035289f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:36.253513 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-53fd5d10-1fcd-4026-a11b-f1476035289f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:36.255006 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-53fd5d10-1fcd-4026-a11b-f1476035289f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:36.256148 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-53fd5d10-1fcd-4026-a11b-f1476035289f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.225100 Jul 05 10:20:36.256725 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 412/851] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:36 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:37.275024 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-fe110d50-2cda-4e99-955b-4dc5e525b385 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:37.275555 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-fe110d50-2cda-4e99-955b-4dc5e525b385 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:37.282445 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fe110d50-2cda-4e99-955b-4dc5e525b385 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:37.282889 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fe110d50-2cda-4e99-955b-4dc5e525b385 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:37.283653 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fe110d50-2cda-4e99-955b-4dc5e525b385 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:37.284082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fe110d50-2cda-4e99-955b-4dc5e525b385 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:37.422001 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-fe110d50-2cda-4e99-955b-4dc5e525b385 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:37.432355 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-fe110d50-2cda-4e99-955b-4dc5e525b385 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:37.433903 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-fe110d50-2cda-4e99-955b-4dc5e525b385 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:37.435089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-fe110d50-2cda-4e99-955b-4dc5e525b385 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.164650 Jul 05 10:20:37.435785 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 440/852] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:37 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:38.453906 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e5d8d9da-34d7-42bf-9691-dd37fe80ffa1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:38.454519 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e5d8d9da-34d7-42bf-9691-dd37fe80ffa1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:38.462132 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e5d8d9da-34d7-42bf-9691-dd37fe80ffa1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:38.462617 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e5d8d9da-34d7-42bf-9691-dd37fe80ffa1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:38.463398 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e5d8d9da-34d7-42bf-9691-dd37fe80ffa1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:38.464063 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e5d8d9da-34d7-42bf-9691-dd37fe80ffa1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:38.603522 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e5d8d9da-34d7-42bf-9691-dd37fe80ffa1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:38.612488 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e5d8d9da-34d7-42bf-9691-dd37fe80ffa1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:38.613971 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e5d8d9da-34d7-42bf-9691-dd37fe80ffa1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:38.615137 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e5d8d9da-34d7-42bf-9691-dd37fe80ffa1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.165210 Jul 05 10:20:38.615705 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 413/853] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:38 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:39.634724 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0bfafeb8-6f78-4801-aff4-460cc2520f7d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:39.635309 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0bfafeb8-6f78-4801-aff4-460cc2520f7d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:39.641587 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0bfafeb8-6f78-4801-aff4-460cc2520f7d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:39.642035 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0bfafeb8-6f78-4801-aff4-460cc2520f7d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:39.642801 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0bfafeb8-6f78-4801-aff4-460cc2520f7d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:39.643236 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0bfafeb8-6f78-4801-aff4-460cc2520f7d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:39.779290 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0bfafeb8-6f78-4801-aff4-460cc2520f7d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:39.788994 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0bfafeb8-6f78-4801-aff4-460cc2520f7d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:39.790839 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0bfafeb8-6f78-4801-aff4-460cc2520f7d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:39.792295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0bfafeb8-6f78-4801-aff4-460cc2520f7d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.161416 Jul 05 10:20:39.793149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 441/854] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:39 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:40.812322 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-bdb19c2a-5f24-4471-b36f-86ac35a40fcf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:40.813094 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bdb19c2a-5f24-4471-b36f-86ac35a40fcf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:40.819818 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bdb19c2a-5f24-4471-b36f-86ac35a40fcf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:40.820281 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bdb19c2a-5f24-4471-b36f-86ac35a40fcf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:40.822300 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bdb19c2a-5f24-4471-b36f-86ac35a40fcf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:40.822751 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bdb19c2a-5f24-4471-b36f-86ac35a40fcf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:40.962387 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-bdb19c2a-5f24-4471-b36f-86ac35a40fcf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:40.972702 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-bdb19c2a-5f24-4471-b36f-86ac35a40fcf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:40.974474 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-bdb19c2a-5f24-4471-b36f-86ac35a40fcf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:40.975627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-bdb19c2a-5f24-4471-b36f-86ac35a40fcf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.167275 Jul 05 10:20:40.976181 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 414/855] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:40 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 05 10:20:41.997229 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d5d0b3fa-e49e-45d6-bbb2-f29f28f9ee9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:41.998487 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d5d0b3fa-e49e-45d6-bbb2-f29f28f9ee9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:42.005394 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d5d0b3fa-e49e-45d6-bbb2-f29f28f9ee9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:42.006118 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d5d0b3fa-e49e-45d6-bbb2-f29f28f9ee9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:42.007083 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d5d0b3fa-e49e-45d6-bbb2-f29f28f9ee9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:42.007704 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d5d0b3fa-e49e-45d6-bbb2-f29f28f9ee9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:42.167308 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d5d0b3fa-e49e-45d6-bbb2-f29f28f9ee9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:42.176034 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d5d0b3fa-e49e-45d6-bbb2-f29f28f9ee9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:42.177451 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d5d0b3fa-e49e-45d6-bbb2-f29f28f9ee9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:42.178678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d5d0b3fa-e49e-45d6-bbb2-f29f28f9ee9a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.187448 Jul 05 10:20:42.179239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 442/856] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:41 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:43.197275 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fff65c3a-d1da-4dea-a24a-77faa25e03df tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:43.197994 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fff65c3a-d1da-4dea-a24a-77faa25e03df tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:43.205023 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fff65c3a-d1da-4dea-a24a-77faa25e03df tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:43.205483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fff65c3a-d1da-4dea-a24a-77faa25e03df tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:43.206267 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fff65c3a-d1da-4dea-a24a-77faa25e03df tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:43.206714 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fff65c3a-d1da-4dea-a24a-77faa25e03df tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:43.352217 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fff65c3a-d1da-4dea-a24a-77faa25e03df tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:43.362599 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fff65c3a-d1da-4dea-a24a-77faa25e03df tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:43.364641 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fff65c3a-d1da-4dea-a24a-77faa25e03df tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:43.366547 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fff65c3a-d1da-4dea-a24a-77faa25e03df tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.173528 Jul 05 10:20:43.368232 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 415/857] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:43 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:44.387153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-eb48bc63-9bfc-48b0-864f-828422ea40fd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:44.387926 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-eb48bc63-9bfc-48b0-864f-828422ea40fd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:44.394850 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-eb48bc63-9bfc-48b0-864f-828422ea40fd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:44.395353 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-eb48bc63-9bfc-48b0-864f-828422ea40fd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:44.396235 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-eb48bc63-9bfc-48b0-864f-828422ea40fd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:44.396800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-eb48bc63-9bfc-48b0-864f-828422ea40fd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:44.539831 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-eb48bc63-9bfc-48b0-864f-828422ea40fd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:44.549420 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-eb48bc63-9bfc-48b0-864f-828422ea40fd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:44.550914 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-eb48bc63-9bfc-48b0-864f-828422ea40fd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:44.552076 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-eb48bc63-9bfc-48b0-864f-828422ea40fd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.169843 Jul 05 10:20:44.552673 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 443/858] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:44 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:45.571086 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2e73949f-7863-48e7-a542-77e0dc71c145 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:45.571649 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2e73949f-7863-48e7-a542-77e0dc71c145 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:45.578887 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2e73949f-7863-48e7-a542-77e0dc71c145 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:45.579368 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2e73949f-7863-48e7-a542-77e0dc71c145 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:45.580135 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2e73949f-7863-48e7-a542-77e0dc71c145 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:45.580611 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2e73949f-7863-48e7-a542-77e0dc71c145 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:45.706254 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-2e73949f-7863-48e7-a542-77e0dc71c145 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:45.715227 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-2e73949f-7863-48e7-a542-77e0dc71c145 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:45.716770 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-2e73949f-7863-48e7-a542-77e0dc71c145 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:45.717918 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2e73949f-7863-48e7-a542-77e0dc71c145 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.150834 Jul 05 10:20:45.718486 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 416/859] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:45 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:46.736244 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-af443ea7-89ae-4a99-8f86-2e5e7c549044 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:46.736876 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-af443ea7-89ae-4a99-8f86-2e5e7c549044 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:46.743646 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-af443ea7-89ae-4a99-8f86-2e5e7c549044 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:46.744093 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-af443ea7-89ae-4a99-8f86-2e5e7c549044 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:46.744882 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-af443ea7-89ae-4a99-8f86-2e5e7c549044 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:46.745321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-af443ea7-89ae-4a99-8f86-2e5e7c549044 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:46.875663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-af443ea7-89ae-4a99-8f86-2e5e7c549044 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:46.884057 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-af443ea7-89ae-4a99-8f86-2e5e7c549044 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:46.886248 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-af443ea7-89ae-4a99-8f86-2e5e7c549044 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:46.887398 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-af443ea7-89ae-4a99-8f86-2e5e7c549044 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.155008 Jul 05 10:20:46.887961 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 444/860] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:46 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:47.878677 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-08913d61-a724-451a-acf1-7104cb055af6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:47.878677 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-08913d61-a724-451a-acf1-7104cb055af6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:47.883570 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-08913d61-a724-451a-acf1-7104cb055af6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:47.884021 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-08913d61-a724-451a-acf1-7104cb055af6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:47.885334 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-08913d61-a724-451a-acf1-7104cb055af6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:47.885770 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-08913d61-a724-451a-acf1-7104cb055af6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:47.905881 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-384b7caa-4ad7-4e69-9cd6-f803732f8ced tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:47.906563 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-384b7caa-4ad7-4e69-9cd6-f803732f8ced tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:47.908786 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-08913d61-a724-451a-acf1-7104cb055af6 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14/ips" status: 200 len: 149 microversion: 2.1 time: 0.168399 Jul 05 10:20:47.909338 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 417/861] 10.222.0.20 () {58 vars in 1247 bytes} [Tue Jul 5 10:20:47 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14/ips => generated 149 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:20:47.913170 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-384b7caa-4ad7-4e69-9cd6-f803732f8ced tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:47.913610 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-384b7caa-4ad7-4e69-9cd6-f803732f8ced tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:47.914658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-384b7caa-4ad7-4e69-9cd6-f803732f8ced tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:47.915093 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-384b7caa-4ad7-4e69-9cd6-f803732f8ced tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:48.051216 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-384b7caa-4ad7-4e69-9cd6-f803732f8ced tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:48.060033 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-384b7caa-4ad7-4e69-9cd6-f803732f8ced tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:48.061591 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-384b7caa-4ad7-4e69-9cd6-f803732f8ced tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:48.062747 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-384b7caa-4ad7-4e69-9cd6-f803732f8ced tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.160750 Jul 05 10:20:48.063440 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 445/862] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:47 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:49.081138 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-93a0ac27-beda-4742-ab83-b14d3e4bcfb1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:49.081752 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-93a0ac27-beda-4742-ab83-b14d3e4bcfb1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:49.088213 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-93a0ac27-beda-4742-ab83-b14d3e4bcfb1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:49.088709 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-93a0ac27-beda-4742-ab83-b14d3e4bcfb1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:49.089478 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-93a0ac27-beda-4742-ab83-b14d3e4bcfb1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:49.089920 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-93a0ac27-beda-4742-ab83-b14d3e4bcfb1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:49.219655 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-93a0ac27-beda-4742-ab83-b14d3e4bcfb1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:49.226636 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-93a0ac27-beda-4742-ab83-b14d3e4bcfb1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:49.227075 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-7bf99ea8-89b7-4bff-9131-a9ad1a2470dd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4fe01c73-dd70-41b9-aa6e-2c1091a900ce"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:20:49.228133 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-93a0ac27-beda-4742-ab83-b14d3e4bcfb1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:49.228267 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7bf99ea8-89b7-4bff-9131-a9ad1a2470dd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:49.229837 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-93a0ac27-beda-4742-ab83-b14d3e4bcfb1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.152582 Jul 05 10:20:49.230387 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 418/863] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:49 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:49.235290 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7bf99ea8-89b7-4bff-9131-a9ad1a2470dd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:49.235745 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7bf99ea8-89b7-4bff-9131-a9ad1a2470dd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:49.236528 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7bf99ea8-89b7-4bff-9131-a9ad1a2470dd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:49.236963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7bf99ea8-89b7-4bff-9131-a9ad1a2470dd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:50.249005 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6d3d97e4-729b-498f-973a-62e0a851c3c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:50.249639 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6d3d97e4-729b-498f-973a-62e0a851c3c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:50.256140 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6d3d97e4-729b-498f-973a-62e0a851c3c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:50.256618 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6d3d97e4-729b-498f-973a-62e0a851c3c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:50.257377 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6d3d97e4-729b-498f-973a-62e0a851c3c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:50.257812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6d3d97e4-729b-498f-973a-62e0a851c3c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:50.398235 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6d3d97e4-729b-498f-973a-62e0a851c3c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:50.407336 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6d3d97e4-729b-498f-973a-62e0a851c3c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:50.408881 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6d3d97e4-729b-498f-973a-62e0a851c3c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:50.410042 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6d3d97e4-729b-498f-973a-62e0a851c3c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.165707 Jul 05 10:20:50.410592 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 419/864] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:50 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 05 10:20:50.428945 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-7bf99ea8-89b7-4bff-9131-a9ad1a2470dd tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "POST /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.205471 Jul 05 10:20:50.429561 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 446/865] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:20:49 2022] POST /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14/os-volume_attachments => generated 194 bytes in 1207 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:20:51.429343 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-18d33b71-b6b7-44a3-89f3-962d7f1b29ab tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:51.429962 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-18d33b71-b6b7-44a3-89f3-962d7f1b29ab tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:51.436509 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-18d33b71-b6b7-44a3-89f3-962d7f1b29ab tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:51.436976 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-18d33b71-b6b7-44a3-89f3-962d7f1b29ab tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:51.437744 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-18d33b71-b6b7-44a3-89f3-962d7f1b29ab tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:51.438191 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-18d33b71-b6b7-44a3-89f3-962d7f1b29ab tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:51.567609 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-18d33b71-b6b7-44a3-89f3-962d7f1b29ab tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:51.576365 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-18d33b71-b6b7-44a3-89f3-962d7f1b29ab tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:51.577882 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-18d33b71-b6b7-44a3-89f3-962d7f1b29ab tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:51.579040 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-18d33b71-b6b7-44a3-89f3-962d7f1b29ab tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.153658 Jul 05 10:20:51.579601 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 420/866] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:51 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:52.598485 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d653e7b8-d51f-4fc8-a6d6-cc7dfb04d210 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:52.599168 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d653e7b8-d51f-4fc8-a6d6-cc7dfb04d210 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:52.605358 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d653e7b8-d51f-4fc8-a6d6-cc7dfb04d210 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:52.605800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d653e7b8-d51f-4fc8-a6d6-cc7dfb04d210 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:52.606547 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d653e7b8-d51f-4fc8-a6d6-cc7dfb04d210 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:52.606976 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d653e7b8-d51f-4fc8-a6d6-cc7dfb04d210 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:52.736162 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d653e7b8-d51f-4fc8-a6d6-cc7dfb04d210 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:52.744736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d653e7b8-d51f-4fc8-a6d6-cc7dfb04d210 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:52.746238 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d653e7b8-d51f-4fc8-a6d6-cc7dfb04d210 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:52.747399 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d653e7b8-d51f-4fc8-a6d6-cc7dfb04d210 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.153555 Jul 05 10:20:52.747963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 447/867] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:52 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:53.765736 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7d95e820-97ff-4bf0-90af-082fa566be81 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:53.766376 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7d95e820-97ff-4bf0-90af-082fa566be81 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:53.773063 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7d95e820-97ff-4bf0-90af-082fa566be81 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:53.773515 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7d95e820-97ff-4bf0-90af-082fa566be81 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:53.774276 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7d95e820-97ff-4bf0-90af-082fa566be81 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:53.774716 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7d95e820-97ff-4bf0-90af-082fa566be81 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:53.908764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-7d95e820-97ff-4bf0-90af-082fa566be81 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:53.917603 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-7d95e820-97ff-4bf0-90af-082fa566be81 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:53.919097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-7d95e820-97ff-4bf0-90af-082fa566be81 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:53.920245 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7d95e820-97ff-4bf0-90af-082fa566be81 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.158388 Jul 05 10:20:53.920820 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 421/868] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:53 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:54.938275 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f6168c66-b9ea-4404-bfa8-823af76acb4b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:54.938879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f6168c66-b9ea-4404-bfa8-823af76acb4b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:54.945436 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f6168c66-b9ea-4404-bfa8-823af76acb4b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:54.945880 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f6168c66-b9ea-4404-bfa8-823af76acb4b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:54.946638 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f6168c66-b9ea-4404-bfa8-823af76acb4b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:54.947079 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f6168c66-b9ea-4404-bfa8-823af76acb4b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:55.074793 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f6168c66-b9ea-4404-bfa8-823af76acb4b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:55.077109 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=90,nova_cell1:INSERT=4,nova_cell1:UPDATE=6 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:20:55.085014 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f6168c66-b9ea-4404-bfa8-823af76acb4b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:55.086495 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f6168c66-b9ea-4404-bfa8-823af76acb4b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:55.087633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f6168c66-b9ea-4404-bfa8-823af76acb4b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.153214 Jul 05 10:20:55.088708 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 448/869] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:54 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:56.106382 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-00c7a28c-5657-4bb6-a115-36896b5001a6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:56.107141 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-00c7a28c-5657-4bb6-a115-36896b5001a6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:56.113623 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-00c7a28c-5657-4bb6-a115-36896b5001a6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:56.114074 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-00c7a28c-5657-4bb6-a115-36896b5001a6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:56.114831 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-00c7a28c-5657-4bb6-a115-36896b5001a6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:56.115263 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-00c7a28c-5657-4bb6-a115-36896b5001a6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:56.245869 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-00c7a28c-5657-4bb6-a115-36896b5001a6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:56.254128 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-00c7a28c-5657-4bb6-a115-36896b5001a6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:56.255623 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-00c7a28c-5657-4bb6-a115-36896b5001a6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:56.256791 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-00c7a28c-5657-4bb6-a115-36896b5001a6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.154382 Jul 05 10:20:56.257541 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 422/870] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:56 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:57.276171 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-31132d5f-e624-4dcb-bda3-33e4ce1cf758 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:57.276816 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-31132d5f-e624-4dcb-bda3-33e4ce1cf758 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:57.284232 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-31132d5f-e624-4dcb-bda3-33e4ce1cf758 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:57.284704 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-31132d5f-e624-4dcb-bda3-33e4ce1cf758 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:57.285455 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-31132d5f-e624-4dcb-bda3-33e4ce1cf758 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:57.285885 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-31132d5f-e624-4dcb-bda3-33e4ce1cf758 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:57.413336 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-31132d5f-e624-4dcb-bda3-33e4ce1cf758 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:57.422187 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-31132d5f-e624-4dcb-bda3-33e4ce1cf758 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:57.423694 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-31132d5f-e624-4dcb-bda3-33e4ce1cf758 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:57.424889 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-31132d5f-e624-4dcb-bda3-33e4ce1cf758 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.153020 Jul 05 10:20:57.425480 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 449/871] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:57 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:58.000420 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-4fc7c2ac-6a33-4d6f-89bc-a2fa7a3dd94e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:58.000938 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-4fc7c2ac-6a33-4d6f-89bc-a2fa7a3dd94e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:58.007338 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4fc7c2ac-6a33-4d6f-89bc-a2fa7a3dd94e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:58.007786 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4fc7c2ac-6a33-4d6f-89bc-a2fa7a3dd94e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:58.008591 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4fc7c2ac-6a33-4d6f-89bc-a2fa7a3dd94e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:58.009033 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4fc7c2ac-6a33-4d6f-89bc-a2fa7a3dd94e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:58.087580 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-4fc7c2ac-6a33-4d6f-89bc-a2fa7a3dd94e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lazy-loading 'services' on Instance uuid d872dbf0-7f48-4887-85d4-9b9d0123be14 {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:20:58.159552 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-4fc7c2ac-6a33-4d6f-89bc-a2fa7a3dd94e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14/os-volume_attachments/4fe01c73-dd70-41b9-aa6e-2c1091a900ce" status: 202 len: 0 microversion: 2.1 time: 0.162902 Jul 05 10:20:58.160155 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 423/872] 10.222.0.20 () {58 vars in 1415 bytes} [Tue Jul 5 10:20:57 2022] DELETE /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14/os-volume_attachments/4fe01c73-dd70-41b9-aa6e-2c1091a900ce => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:20:58.169120 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-60c251ba-5fcf-4c34-a1ab-12e676f86213 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:58.170096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-60c251ba-5fcf-4c34-a1ab-12e676f86213 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:58.175986 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-60c251ba-5fcf-4c34-a1ab-12e676f86213 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:58.176541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-60c251ba-5fcf-4c34-a1ab-12e676f86213 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:58.177217 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-60c251ba-5fcf-4c34-a1ab-12e676f86213 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:58.177687 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-60c251ba-5fcf-4c34-a1ab-12e676f86213 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:58.207761 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-60c251ba-5fcf-4c34-a1ab-12e676f86213 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043194 Jul 05 10:20:58.208325 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 450/873] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:20:58 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:20:58.443430 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-de04344e-a31e-49e7-941a-55dd12a3930b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:58.444158 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-de04344e-a31e-49e7-941a-55dd12a3930b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:58.450936 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-de04344e-a31e-49e7-941a-55dd12a3930b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:58.451388 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-de04344e-a31e-49e7-941a-55dd12a3930b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:58.452150 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-de04344e-a31e-49e7-941a-55dd12a3930b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:58.452618 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-de04344e-a31e-49e7-941a-55dd12a3930b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:58.585494 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-de04344e-a31e-49e7-941a-55dd12a3930b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:58.593982 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-de04344e-a31e-49e7-941a-55dd12a3930b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:58.595519 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-de04344e-a31e-49e7-941a-55dd12a3930b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:58.596705 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-de04344e-a31e-49e7-941a-55dd12a3930b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.157490 Jul 05 10:20:58.602789 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 424/874] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:58 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:20:59.219790 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-efe53b28-bf01-4f10-ba6f-ae907300a7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:59.221014 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-efe53b28-bf01-4f10-ba6f-ae907300a7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:59.227398 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-efe53b28-bf01-4f10-ba6f-ae907300a7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:59.227852 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-efe53b28-bf01-4f10-ba6f-ae907300a7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:59.228653 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-efe53b28-bf01-4f10-ba6f-ae907300a7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:59.229124 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-efe53b28-bf01-4f10-ba6f-ae907300a7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:59.257640 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-efe53b28-bf01-4f10-ba6f-ae907300a7f0 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041717 Jul 05 10:20:59.258120 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 451/875] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:20:59 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 05 10:20:59.314129 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-848ff587-d10e-4e33-9315-b19bbeaa69e1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:59.314634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-848ff587-d10e-4e33-9315-b19bbeaa69e1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:59.320999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-848ff587-d10e-4e33-9315-b19bbeaa69e1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:59.321474 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-848ff587-d10e-4e33-9315-b19bbeaa69e1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:59.322229 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-848ff587-d10e-4e33-9315-b19bbeaa69e1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:59.322679 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-848ff587-d10e-4e33-9315-b19bbeaa69e1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:59.326663 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:20:59.348275 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-848ff587-d10e-4e33-9315-b19bbeaa69e1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Going to try to terminate instance {{(pid=117812) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:20:59.422259 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-848ff587-d10e-4e33-9315-b19bbeaa69e1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 204 len: 0 microversion: 2.1 time: 0.111671 Jul 05 10:20:59.422980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 425/876] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:20:59 2022] DELETE /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:20:59.432636 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-7ed538bc-b310-48d3-a05d-de0e0851ece5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:59.433249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7ed538bc-b310-48d3-a05d-de0e0851ece5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:59.439215 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7ed538bc-b310-48d3-a05d-de0e0851ece5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:59.439668 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7ed538bc-b310-48d3-a05d-de0e0851ece5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:59.440485 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7ed538bc-b310-48d3-a05d-de0e0851ece5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:59.440933 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7ed538bc-b310-48d3-a05d-de0e0851ece5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:59.567514 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-7ed538bc-b310-48d3-a05d-de0e0851ece5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:59.575548 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-7ed538bc-b310-48d3-a05d-de0e0851ece5 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1682 microversion: 2.1 time: 0.147203 Jul 05 10:20:59.576116 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 452/877] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:20:59 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1682 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:20:59.615115 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ccb305b7-7d5e-4cf7-b1b7-62f988a5f8fc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:20:59.615805 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ccb305b7-7d5e-4cf7-b1b7-62f988a5f8fc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:20:59.622531 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ccb305b7-7d5e-4cf7-b1b7-62f988a5f8fc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:59.622980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ccb305b7-7d5e-4cf7-b1b7-62f988a5f8fc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:59.623739 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ccb305b7-7d5e-4cf7-b1b7-62f988a5f8fc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:20:59.624177 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ccb305b7-7d5e-4cf7-b1b7-62f988a5f8fc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:20:59.904805 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ccb305b7-7d5e-4cf7-b1b7-62f988a5f8fc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:59.913177 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ccb305b7-7d5e-4cf7-b1b7-62f988a5f8fc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:59.914665 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ccb305b7-7d5e-4cf7-b1b7-62f988a5f8fc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:20:59.915807 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ccb305b7-7d5e-4cf7-b1b7-62f988a5f8fc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.304887 Jul 05 10:20:59.916358 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 426/878] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:20:59 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:21:00.592735 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5263ed95-3f16-4600-8df8-4ee232e561f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:00.593517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5263ed95-3f16-4600-8df8-4ee232e561f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:00.600295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5263ed95-3f16-4600-8df8-4ee232e561f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:00.600766 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5263ed95-3f16-4600-8df8-4ee232e561f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:00.601528 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5263ed95-3f16-4600-8df8-4ee232e561f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:00.601956 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5263ed95-3f16-4600-8df8-4ee232e561f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:00.651613 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-393413a0-ebd5-4c30-b6df-10fed825d51b req-5b51dc5f-346e-4e33-9cb7-7edc22ba2132 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ff442846-3ded-48c4-8a48-df125f46d0d6", "name": "network-vif-plugged", "status": "completed", "tag": "e6fb0593-584d-4308-86f5-4705f3dc20c8"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:21:00.661417 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-393413a0-ebd5-4c30-b6df-10fed825d51b req-5b51dc5f-346e-4e33-9cb7-7edc22ba2132 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:00.663025 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-393413a0-ebd5-4c30-b6df-10fed825d51b req-5b51dc5f-346e-4e33-9cb7-7edc22ba2132 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:00.675346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-393413a0-ebd5-4c30-b6df-10fed825d51b req-5b51dc5f-346e-4e33-9cb7-7edc22ba2132 service nova] Creating event network-vif-plugged:e6fb0593-584d-4308-86f5-4705f3dc20c8 for instance ff442846-3ded-48c4-8a48-df125f46d0d6 on hv-cinder-maste Jul 05 10:21:00.680256 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-393413a0-ebd5-4c30-b6df-10fed825d51b req-5b51dc5f-346e-4e33-9cb7-7edc22ba2132 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032633 Jul 05 10:21:00.681102 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 427/879] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:21:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:21:00.738413 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5263ed95-3f16-4600-8df8-4ee232e561f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:00.747035 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5263ed95-3f16-4600-8df8-4ee232e561f9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1682 microversion: 2.1 time: 0.158145 Jul 05 10:21:00.747594 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 453/880] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:21:00 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1682 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:21:00.935675 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-19fa6510-81cd-4379-855c-38e779d185b1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:00.936454 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-19fa6510-81cd-4379-855c-38e779d185b1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:00.943036 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-19fa6510-81cd-4379-855c-38e779d185b1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:00.943477 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-19fa6510-81cd-4379-855c-38e779d185b1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:00.944233 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-19fa6510-81cd-4379-855c-38e779d185b1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:00.944689 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-19fa6510-81cd-4379-855c-38e779d185b1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:01.074101 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-19fa6510-81cd-4379-855c-38e779d185b1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:01.082452 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-19fa6510-81cd-4379-855c-38e779d185b1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:01.083935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-19fa6510-81cd-4379-855c-38e779d185b1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:01.085162 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-19fa6510-81cd-4379-855c-38e779d185b1 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.154966 Jul 05 10:21:01.085710 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 428/881] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:21:00 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:21:01.765119 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-890cd94e-0b09-46f7-9490-fc4c1d060985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:01.765760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-890cd94e-0b09-46f7-9490-fc4c1d060985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:01.772123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-890cd94e-0b09-46f7-9490-fc4c1d060985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:01.772604 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-890cd94e-0b09-46f7-9490-fc4c1d060985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:01.773369 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-890cd94e-0b09-46f7-9490-fc4c1d060985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:01.773827 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-890cd94e-0b09-46f7-9490-fc4c1d060985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:01.993249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-890cd94e-0b09-46f7-9490-fc4c1d060985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:02.001707 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-890cd94e-0b09-46f7-9490-fc4c1d060985 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1682 microversion: 2.1 time: 0.241004 Jul 05 10:21:02.002665 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 454/882] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:21:01 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1682 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:21:02.103063 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f390142c-93bd-4f73-a6c0-01ebccfb3c87 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:02.103798 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f390142c-93bd-4f73-a6c0-01ebccfb3c87 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:02.110220 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f390142c-93bd-4f73-a6c0-01ebccfb3c87 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:02.110680 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f390142c-93bd-4f73-a6c0-01ebccfb3c87 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:02.111455 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f390142c-93bd-4f73-a6c0-01ebccfb3c87 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:02.111899 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f390142c-93bd-4f73-a6c0-01ebccfb3c87 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:02.342173 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f390142c-93bd-4f73-a6c0-01ebccfb3c87 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:02.350436 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f390142c-93bd-4f73-a6c0-01ebccfb3c87 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:02.351929 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f390142c-93bd-4f73-a6c0-01ebccfb3c87 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:02.353095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f390142c-93bd-4f73-a6c0-01ebccfb3c87 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.253987 Jul 05 10:21:02.353651 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 429/883] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:21:02 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:21:02.590589 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-795275f9-66e0-49b2-9005-baa4eecc8896 req-af5294c8-2ef1-49f6-9d00-93e671f2fb45 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d872dbf0-7f48-4887-85d4-9b9d0123be14", "name": "network-vif-deleted", "tag": "705d8fd5-bec7-4c13-8533-3ab583c1b1bb"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:21:02.594212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=38 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:21:02.601689 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-795275f9-66e0-49b2-9005-baa4eecc8896 req-af5294c8-2ef1-49f6-9d00-93e671f2fb45 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:02.602418 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-795275f9-66e0-49b2-9005-baa4eecc8896 req-af5294c8-2ef1-49f6-9d00-93e671f2fb45 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:02.614708 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-795275f9-66e0-49b2-9005-baa4eecc8896 req-af5294c8-2ef1-49f6-9d00-93e671f2fb45 service nova] Creating event network-vif-deleted:705d8fd5-bec7-4c13-8533-3ab583c1b1bb for instance d872dbf0-7f48-4887-85d4-9b9d0123be14 on hv-cinder-maste Jul 05 10:21:02.627055 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-795275f9-66e0-49b2-9005-baa4eecc8896 req-af5294c8-2ef1-49f6-9d00-93e671f2fb45 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.119725 Jul 05 10:21:02.628075 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 455/884] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:21:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:21:03.019001 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7836fea9-861f-4236-b454-87a084eacdb9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:03.019610 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7836fea9-861f-4236-b454-87a084eacdb9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:03.025872 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7836fea9-861f-4236-b454-87a084eacdb9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:03.026319 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7836fea9-861f-4236-b454-87a084eacdb9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:03.027100 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7836fea9-861f-4236-b454-87a084eacdb9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:03.027534 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7836fea9-861f-4236-b454-87a084eacdb9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:03.064685 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:21:03.082215 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7836fea9-861f-4236-b454-87a084eacdb9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:03.082616 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7836fea9-861f-4236-b454-87a084eacdb9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:03.083085 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-7836fea9-861f-4236-b454-87a084eacdb9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lazy-loading 'fault' on Instance uuid d872dbf0-7f48-4887-85d4-9b9d0123be14 {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:21:03.147280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-7836fea9-861f-4236-b454-87a084eacdb9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '036cb887c7894e75ba3eb4c1a8b21c69', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7eccead420ae4aacb6f31dcb17a138cc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:03.155157 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7836fea9-861f-4236-b454-87a084eacdb9 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 200 len: 1332 microversion: 2.1 time: 0.139916 Jul 05 10:21:03.155735 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 430/885] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:21:03 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 1332 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:21:03.372645 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-910e712b-5cdf-436e-919a-e1fceae87578 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:03.373475 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-910e712b-5cdf-436e-919a-e1fceae87578 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:03.379819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-910e712b-5cdf-436e-919a-e1fceae87578 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:03.380344 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-910e712b-5cdf-436e-919a-e1fceae87578 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:03.381220 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-910e712b-5cdf-436e-919a-e1fceae87578 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:03.381729 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-910e712b-5cdf-436e-919a-e1fceae87578 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:03.527214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-910e712b-5cdf-436e-919a-e1fceae87578 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:03.542972 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-910e712b-5cdf-436e-919a-e1fceae87578 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:03.546302 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-910e712b-5cdf-436e-919a-e1fceae87578 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:03.548627 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-910e712b-5cdf-436e-919a-e1fceae87578 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1808 microversion: 2.20 time: 0.181175 Jul 05 10:21:03.549536 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 456/886] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:21:03 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1808 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:21:04.172877 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7b7e2164-1804-49e2-9849-a455057ba222 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:04.173491 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7b7e2164-1804-49e2-9849-a455057ba222 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:04.179968 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7b7e2164-1804-49e2-9849-a455057ba222 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:04.180453 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7b7e2164-1804-49e2-9849-a455057ba222 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:04.181230 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7b7e2164-1804-49e2-9849-a455057ba222 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:04.181670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7b7e2164-1804-49e2-9849-a455057ba222 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:04.195159 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-7b7e2164-1804-49e2-9849-a455057ba222 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7b7e2164-1804-49e2-9849-a455057ba222 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 HTTP/1.1 Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:21:04.196813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:21:04.198487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:21:04.198487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:04.198487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:04.198487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:04.198487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:04.198487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:21:04.198487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:21:04.198487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:21:04.198487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:04.198487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:21:04.198487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:04.198487 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': 'd872dbf0-7f48-4887-85d4-9b9d0123be14'} Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:21:04.199356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:04.201394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:04.202859 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7b7e2164-1804-49e2-9849-a455057ba222 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 404 to user: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:21:04.202859 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7b7e2164-1804-49e2-9849-a455057ba222 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 404 len: 111 microversion: 2.1 time: 0.029159 Jul 05 10:21:04.202859 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 431/887] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:21:04 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:21:04.207027 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-942132e9-86e1-4efe-9929-f412a5ae0e9e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '.version_select at 0x7feb8df03af0>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:04.224158 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-942132e9-86e1-4efe-9929-f412a5ae0e9e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1306729611" status: 202 len: 0 microversion: 2.1 time: 0.020817 Jul 05 10:21:04.224747 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 457/888] 10.222.0.20 () {58 vars in 1220 bytes} [Tue Jul 5 10:21:04 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1306729611 => generated 0 bytes in 22 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:21:04.567218 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fae95cbd-303e-4c8b-9f10-a5fccf0024cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:04.567847 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fae95cbd-303e-4c8b-9f10-a5fccf0024cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:04.574107 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fae95cbd-303e-4c8b-9f10-a5fccf0024cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:04.574554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fae95cbd-303e-4c8b-9f10-a5fccf0024cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:04.575336 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fae95cbd-303e-4c8b-9f10-a5fccf0024cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:04.575790 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fae95cbd-303e-4c8b-9f10-a5fccf0024cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:04.939420 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fae95cbd-303e-4c8b-9f10-a5fccf0024cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:04.948259 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fae95cbd-303e-4c8b-9f10-a5fccf0024cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:04.949803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fae95cbd-303e-4c8b-9f10-a5fccf0024cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:21:04.950997 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fae95cbd-303e-4c8b-9f10-a5fccf0024cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1795 microversion: 2.20 time: 0.387695 Jul 05 10:21:04.951544 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 432/889] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:21:04 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1795 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:21:06.247247 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d8f84e91-5631-4fa8-a9f9-2dc26e979a5a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:06.247760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d8f84e91-5631-4fa8-a9f9-2dc26e979a5a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:06.254283 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8f84e91-5631-4fa8-a9f9-2dc26e979a5a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:06.254724 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8f84e91-5631-4fa8-a9f9-2dc26e979a5a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:06.255502 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8f84e91-5631-4fa8-a9f9-2dc26e979a5a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:06.255930 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8f84e91-5631-4fa8-a9f9-2dc26e979a5a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:06.269632 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-d8f84e91-5631-4fa8-a9f9-2dc26e979a5a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d8f84e91-5631-4fa8-a9f9-2dc26e979a5a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: DELETE /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 HTTP/1.1 Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:21:06.271289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:21:06.272887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:21:06.272887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:06.272887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:06.272887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:06.272887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:06.272887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:21:06.272887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:21:06.272887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:21:06.272887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:06.272887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:21:06.272887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:06.272887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'd872dbf0-7f48-4887-85d4-9b9d0123be14'} Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:21:06.273746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:06.275153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:06.277102 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d8f84e91-5631-4fa8-a9f9-2dc26e979a5a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 404 to user: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:21:06.277102 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d8f84e91-5631-4fa8-a9f9-2dc26e979a5a tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 404 len: 111 microversion: 2.1 time: 0.030148 Jul 05 10:21:06.277102 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 458/890] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:21:06 2022] DELETE /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:21:06.281438 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2daab377-ee44-4c5a-a897-5bfaefcfb929 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:06.282039 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2daab377-ee44-4c5a-a897-5bfaefcfb929 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: d872dbf0-7f48-4887-85d4-9b9d0123be14] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:06.288483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2daab377-ee44-4c5a-a897-5bfaefcfb929 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:06.288933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2daab377-ee44-4c5a-a897-5bfaefcfb929 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:06.289729 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2daab377-ee44-4c5a-a897-5bfaefcfb929 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:06.290162 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2daab377-ee44-4c5a-a897-5bfaefcfb929 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:06.303702 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-2daab377-ee44-4c5a-a897-5bfaefcfb929 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2daab377-ee44-4c5a-a897-5bfaefcfb929 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 HTTP/1.1 Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:21:06.305214 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:21:06.306830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:21:06.306830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:06.306830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:06.306830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:06.306830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:06.306830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:21:06.306830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:21:06.306830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:21:06.306830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:06.306830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:21:06.306830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:06.306830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': 'd872dbf0-7f48-4887-85d4-9b9d0123be14'} Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:21:06.307745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. Jul 05 10:21:06.309205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:06.310496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2daab377-ee44-4c5a-a897-5bfaefcfb929 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 404 to user: Instance d872dbf0-7f48-4887-85d4-9b9d0123be14 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:21:06.310496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2daab377-ee44-4c5a-a897-5bfaefcfb929 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14" status: 404 len: 111 microversion: 2.1 time: 0.028919 Jul 05 10:21:06.310496 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 433/891] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:21:06 2022] GET /compute/v2.1/servers/d872dbf0-7f48-4887-85d4-9b9d0123be14 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:21:07.438043 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e05149a7-611d-402f-9ad7-55af93b30ad1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:07.438578 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e05149a7-611d-402f-9ad7-55af93b30ad1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:07.445027 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e05149a7-611d-402f-9ad7-55af93b30ad1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:07.445477 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e05149a7-611d-402f-9ad7-55af93b30ad1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:07.446319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e05149a7-611d-402f-9ad7-55af93b30ad1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:07.446711 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e05149a7-611d-402f-9ad7-55af93b30ad1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:07.460165 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-e05149a7-611d-402f-9ad7-55af93b30ad1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e05149a7-611d-402f-9ad7-55af93b30ad1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: DELETE /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 HTTP/1.1 Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:21:07.461819 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:21:07.463388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:21:07.463388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:07.463388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:07.463388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:07.463388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:07.463388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:21:07.463388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:21:07.463388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:21:07.463388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:07.463388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:21:07.463388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:21:07.463388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': '70cdd8c1-3058-4809-a0d1-73e39cf2e520'} Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:21:07.464800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:21:07.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.467399 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e05149a7-611d-402f-9ad7-55af93b30ad1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 404 to user: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:21:07.467399 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e05149a7-611d-402f-9ad7-55af93b30ad1 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 404 len: 111 microversion: 2.1 time: 0.029073 Jul 05 10:21:07.467399 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 459/892] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:21:07 2022] DELETE /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:21:07.471911 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7c429786-e381-48a7-a467-b68898eeae1e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:07.472561 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7c429786-e381-48a7-a467-b68898eeae1e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: 70cdd8c1-3058-4809-a0d1-73e39cf2e520] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:07.479004 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7c429786-e381-48a7-a467-b68898eeae1e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:07.479961 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7c429786-e381-48a7-a467-b68898eeae1e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:07.480765 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7c429786-e381-48a7-a467-b68898eeae1e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:07.481205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7c429786-e381-48a7-a467-b68898eeae1e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:07.494217 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-7c429786-e381-48a7-a467-b68898eeae1e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7c429786-e381-48a7-a467-b68898eeae1e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 HTTP/1.1 Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:21:07.496773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:21:07.498383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:21:07.498383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:07.498383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:07.498383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:07.498383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:07.498383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:21:07.498383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:21:07.498383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:21:07.498383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:07.498383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:21:07.498383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:21:07.498383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': '70cdd8c1-3058-4809-a0d1-73e39cf2e520'} Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:21:07.499196 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. Jul 05 10:21:07.500668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.502035 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7c429786-e381-48a7-a467-b68898eeae1e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 404 to user: Instance 70cdd8c1-3058-4809-a0d1-73e39cf2e520 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:21:07.502035 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7c429786-e381-48a7-a467-b68898eeae1e tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520" status: 404 len: 111 microversion: 2.1 time: 0.030200 Jul 05 10:21:07.502035 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 434/893] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:21:07 2022] GET /compute/v2.1/servers/70cdd8c1-3058-4809-a0d1-73e39cf2e520 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:21:07.506664 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-accd9a9b-ddec-4815-abae-df8f5aaf6a92 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:07.507159 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-accd9a9b-ddec-4815-abae-df8f5aaf6a92 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:07.513183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-accd9a9b-ddec-4815-abae-df8f5aaf6a92 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:07.513664 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-accd9a9b-ddec-4815-abae-df8f5aaf6a92 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:07.514453 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-accd9a9b-ddec-4815-abae-df8f5aaf6a92 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:07.514921 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-accd9a9b-ddec-4815-abae-df8f5aaf6a92 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:07.528356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-accd9a9b-ddec-4815-abae-df8f5aaf6a92 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-accd9a9b-ddec-4815-abae-df8f5aaf6a92 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: DELETE /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 HTTP/1.1 Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:21:07.529886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:21:07.531396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:21:07.531396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:07.531396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:07.531396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:07.531396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:07.531396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:21:07.531396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:21:07.531396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:21:07.531396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:07.531396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:21:07.531396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:21:07.531396 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'a325da12-82e8-4964-a4db-4d38efa52dc7'} Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:21:07.532415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:21:07.533879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.535395 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-accd9a9b-ddec-4815-abae-df8f5aaf6a92 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 404 to user: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:21:07.535395 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-accd9a9b-ddec-4815-abae-df8f5aaf6a92 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 404 len: 111 microversion: 2.1 time: 0.028318 Jul 05 10:21:07.535395 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 460/894] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:21:07 2022] DELETE /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:21:07.539311 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8fa46bff-bd6a-40c5-873d-dd566da84b0f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:07.539923 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-8fa46bff-bd6a-40c5-873d-dd566da84b0f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: a325da12-82e8-4964-a4db-4d38efa52dc7] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:07.546514 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8fa46bff-bd6a-40c5-873d-dd566da84b0f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:07.546993 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8fa46bff-bd6a-40c5-873d-dd566da84b0f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:07.547755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8fa46bff-bd6a-40c5-873d-dd566da84b0f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:07.548195 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8fa46bff-bd6a-40c5-873d-dd566da84b0f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:07.560983 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-8fa46bff-bd6a-40c5-873d-dd566da84b0f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8fa46bff-bd6a-40c5-873d-dd566da84b0f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 HTTP/1.1 Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:21:07.562470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:21:07.564110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:21:07.564110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:07.564110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:07.564110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:07.564110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:07.564110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:21:07.564110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:21:07.564110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:21:07.564110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:07.564110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:21:07.564110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:21:07.564110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': 'a325da12-82e8-4964-a4db-4d38efa52dc7'} Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:21:07.565114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. Jul 05 10:21:07.566540 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:07.568072 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8fa46bff-bd6a-40c5-873d-dd566da84b0f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 404 to user: Instance a325da12-82e8-4964-a4db-4d38efa52dc7 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:21:07.568072 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-8fa46bff-bd6a-40c5-873d-dd566da84b0f tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7" status: 404 len: 111 microversion: 2.1 time: 0.028260 Jul 05 10:21:07.568072 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 435/895] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:21:07 2022] GET /compute/v2.1/servers/a325da12-82e8-4964-a4db-4d38efa52dc7 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 05 10:21:08.693130 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0590fdd5-8584-4f7a-a0fd-823ea4950e36 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:08.693621 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0590fdd5-8584-4f7a-a0fd-823ea4950e36 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:08.700562 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0590fdd5-8584-4f7a-a0fd-823ea4950e36 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:08.701014 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0590fdd5-8584-4f7a-a0fd-823ea4950e36 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:08.701771 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0590fdd5-8584-4f7a-a0fd-823ea4950e36 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:08.702204 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0590fdd5-8584-4f7a-a0fd-823ea4950e36 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:08.715761 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-0590fdd5-8584-4f7a-a0fd-823ea4950e36 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0590fdd5-8584-4f7a-a0fd-823ea4950e36 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: DELETE /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b HTTP/1.1 Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:21:08.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:21:08.718879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:21:08.718879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:08.718879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:08.718879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:08.718879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:08.718879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:21:08.718879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:21:08.718879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:21:08.718879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:08.718879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:21:08.718879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:21:08.718879 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'ab80efc4-b2fa-4761-806b-2a197cc5b00b'} Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:21:08.719722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:21:08.721096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:21:08.722471 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0590fdd5-8584-4f7a-a0fd-823ea4950e36 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 404 to user: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:21:08.722471 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0590fdd5-8584-4f7a-a0fd-823ea4950e36 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "DELETE /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 404 len: 111 microversion: 2.1 time: 0.029312 Jul 05 10:21:08.722471 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 461/896] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:21:08 2022] DELETE /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:21:08.727706 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-90a0d3b8-2948-4bce-a9b5-0d3d8ff5f339 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:21:08.728320 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-90a0d3b8-2948-4bce-a9b5-0d3d8ff5f339 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] [instance: ab80efc4-b2fa-4761-806b-2a197cc5b00b] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:21:08.734950 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-90a0d3b8-2948-4bce-a9b5-0d3d8ff5f339 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:08.735404 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-90a0d3b8-2948-4bce-a9b5-0d3d8ff5f339 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:08.736175 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-90a0d3b8-2948-4bce-a9b5-0d3d8ff5f339 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:21:08.736660 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-90a0d3b8-2948-4bce-a9b5-0d3d8ff5f339 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:21:08.749956 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-90a0d3b8-2948-4bce-a9b5-0d3d8ff5f339 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] HTTP exception thrown: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-90a0d3b8-2948-4bce-a9b5-0d3d8ff5f339 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Request method failure captured: Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b HTTP/1.1 Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA4LIqS0gAHzNtwS-VpRYzGva6dCkbp0o_ThV8iUQDh89UAJvlKJK0CL6DJMOrLMnl6t45TvVzXSpg1_KLyfVJk3DKgqs-ZSG6icT32IK61sdZdyPTXOxRq2a8HBlLLYy-yYZ_HtZq0-t_L1XWyOTqYtQ04v6XqwIRV-fD5nvISZUIU Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:21:08.751462 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:21:08.753071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/7eccead420ae4aacb6f31dcb17a138cc"}]}] Jul 05 10:21:08.753071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:08.753071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: 7eccead420ae4aacb6f31dcb17a138cc Jul 05 10:21:08.753071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1772833869 Jul 05 10:21:08.753071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:08.753071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:21:08.753071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:21:08.753071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: 036cb887c7894e75ba3eb4c1a8b21c69 Jul 05 10:21:08.753071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-AttachVolumeNegativeTest-1772833869-project Jul 05 10:21:08.753071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:21:08.753071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:21:08.753071 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': 'ab80efc4-b2fa-4761-806b-2a197cc5b00b'} Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:21:08.757165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. Jul 05 10:21:08.758838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:21:08.759909 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-90a0d3b8-2948-4bce-a9b5-0d3d8ff5f339 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] Returning 404 to user: Instance ab80efc4-b2fa-4761-806b-2a197cc5b00b could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:21:08.759909 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-90a0d3b8-2948-4bce-a9b5-0d3d8ff5f339 tempest-AttachVolumeNegativeTest-1772833869 tempest-AttachVolumeNegativeTest-1772833869-project] 10.222.0.20 "GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b" status: 404 len: 111 microversion: 2.1 time: 0.029023 Jul 05 10:21:08.759909 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 436/897] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:21:08 2022] GET /compute/v2.1/servers/ab80efc4-b2fa-4761-806b-2a197cc5b00b => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:21:56.395677 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:21:56.396201 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Tue Jul 5 10:21:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:22:09.124420 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=14,nova_api:DELETE=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:09.125173 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=53 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:09.131110 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-14d71779-fd98-4dca-aa55-6ce26ca837e8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:09.131651 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-14d71779-fd98-4dca-aa55-6ce26ca837e8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:09.139735 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-14d71779-fd98-4dca-aa55-6ce26ca837e8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:09.140183 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-14d71779-fd98-4dca-aa55-6ce26ca837e8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:09.140976 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-14d71779-fd98-4dca-aa55-6ce26ca837e8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:09.141409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-14d71779-fd98-4dca-aa55-6ce26ca837e8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:09.169796 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-14d71779-fd98-4dca-aa55-6ce26ca837e8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments/b0817488-9309-47d3-8022-3cfc6627eefb" status: 200 len: 194 microversion: 2.20 time: 0.047125 Jul 05 10:22:09.170365 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 462/898] 10.222.0.20 () {60 vars in 1453 bytes} [Tue Jul 5 10:22:09 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments/b0817488-9309-47d3-8022-3cfc6627eefb => generated 194 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:22:09.230029 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=13 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:09.230780 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=57,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:09.235127 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e57a8b40-8fd9-4346-96fa-6c7dc3d811a5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:09.235650 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e57a8b40-8fd9-4346-96fa-6c7dc3d811a5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:09.243244 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e57a8b40-8fd9-4346-96fa-6c7dc3d811a5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:09.243701 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e57a8b40-8fd9-4346-96fa-6c7dc3d811a5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:09.244504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e57a8b40-8fd9-4346-96fa-6c7dc3d811a5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:09.244949 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e57a8b40-8fd9-4346-96fa-6c7dc3d811a5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:09.322919 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-e57a8b40-8fd9-4346-96fa-6c7dc3d811a5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lazy-loading 'services' on Instance uuid ff442846-3ded-48c4-8a48-df125f46d0d6 {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:22:09.392556 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e57a8b40-8fd9-4346-96fa-6c7dc3d811a5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "DELETE /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments/b0817488-9309-47d3-8022-3cfc6627eefb" status: 202 len: 0 microversion: 2.20 time: 0.164080 Jul 05 10:22:09.393137 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 437/899] 10.222.0.20 () {60 vars in 1456 bytes} [Tue Jul 5 10:22:09 2022] DELETE /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments/b0817488-9309-47d3-8022-3cfc6627eefb => generated 0 bytes in 165 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:22:09.402060 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-151605e3-459d-4a25-8924-aeecbf0c2baf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:09.403006 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-151605e3-459d-4a25-8924-aeecbf0c2baf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:09.409115 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-151605e3-459d-4a25-8924-aeecbf0c2baf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:09.409517 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-151605e3-459d-4a25-8924-aeecbf0c2baf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:09.410289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-151605e3-459d-4a25-8924-aeecbf0c2baf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:09.410762 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-151605e3-459d-4a25-8924-aeecbf0c2baf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:09.440750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-151605e3-459d-4a25-8924-aeecbf0c2baf tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043006 Jul 05 10:22:09.441232 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 463/900] 10.222.0.20 () {60 vars in 1342 bytes} [Tue Jul 5 10:22:09 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:22:10.453612 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0cb8b767-5b0b-4495-a100-af84a9d74f27 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:10.454670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0cb8b767-5b0b-4495-a100-af84a9d74f27 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:10.460907 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0cb8b767-5b0b-4495-a100-af84a9d74f27 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:10.461357 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0cb8b767-5b0b-4495-a100-af84a9d74f27 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:10.462119 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0cb8b767-5b0b-4495-a100-af84a9d74f27 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:10.462557 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0cb8b767-5b0b-4495-a100-af84a9d74f27 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:10.491516 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0cb8b767-5b0b-4495-a100-af84a9d74f27 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042701 Jul 05 10:22:10.492859 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 438/901] 10.222.0.20 () {60 vars in 1342 bytes} [Tue Jul 5 10:22:10 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:22:11.503511 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6c59d6cb-4e93-4bcf-b071-66a92a77f8dc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:11.504609 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-6c59d6cb-4e93-4bcf-b071-66a92a77f8dc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:11.511592 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6c59d6cb-4e93-4bcf-b071-66a92a77f8dc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:11.512044 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6c59d6cb-4e93-4bcf-b071-66a92a77f8dc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:11.512853 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6c59d6cb-4e93-4bcf-b071-66a92a77f8dc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:11.513296 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6c59d6cb-4e93-4bcf-b071-66a92a77f8dc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:11.542699 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-6c59d6cb-4e93-4bcf-b071-66a92a77f8dc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043490 Jul 05 10:22:11.543262 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 464/902] 10.222.0.20 () {60 vars in 1342 bytes} [Tue Jul 5 10:22:11 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:22:12.555393 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-bfcabbc9-65a6-41c0-80df-32b425f2efb8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:12.556412 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bfcabbc9-65a6-41c0-80df-32b425f2efb8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:12.562783 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bfcabbc9-65a6-41c0-80df-32b425f2efb8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:12.563233 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bfcabbc9-65a6-41c0-80df-32b425f2efb8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:12.564006 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bfcabbc9-65a6-41c0-80df-32b425f2efb8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:12.564470 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bfcabbc9-65a6-41c0-80df-32b425f2efb8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:12.592910 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-bfcabbc9-65a6-41c0-80df-32b425f2efb8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041474 Jul 05 10:22:12.593461 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 439/903] 10.222.0.20 () {60 vars in 1342 bytes} [Tue Jul 5 10:22:12 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:22:13.604585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-bf320e8a-b9c7-465c-8d7e-4d45f730b80c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:13.605493 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-bf320e8a-b9c7-465c-8d7e-4d45f730b80c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:13.612295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bf320e8a-b9c7-465c-8d7e-4d45f730b80c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:13.612774 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bf320e8a-b9c7-465c-8d7e-4d45f730b80c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:13.613671 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bf320e8a-b9c7-465c-8d7e-4d45f730b80c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:13.613941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bf320e8a-b9c7-465c-8d7e-4d45f730b80c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:13.642515 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-bf320e8a-b9c7-465c-8d7e-4d45f730b80c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.042746 Jul 05 10:22:13.643054 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 465/904] 10.222.0.20 () {60 vars in 1342 bytes} [Tue Jul 5 10:22:13 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:22:13.700362 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-245ef502-0147-4f69-bebc-795917396b57 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:13.700897 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-245ef502-0147-4f69-bebc-795917396b57 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:13.707396 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-245ef502-0147-4f69-bebc-795917396b57 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:13.707849 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-245ef502-0147-4f69-bebc-795917396b57 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:13.708711 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-245ef502-0147-4f69-bebc-795917396b57 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:13.709162 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-245ef502-0147-4f69-bebc-795917396b57 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:13.735365 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-245ef502-0147-4f69-bebc-795917396b57 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Going to try to terminate instance {{(pid=117812) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:22:13.813506 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-245ef502-0147-4f69-bebc-795917396b57 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "DELETE /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 204 len: 0 microversion: 2.20 time: 0.116569 Jul 05 10:22:13.814099 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 440/905] 10.222.0.20 () {60 vars in 1279 bytes} [Tue Jul 5 10:22:13 2022] DELETE /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Jul 05 10:22:13.822335 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-18319a1c-04c6-400e-bd4b-0bf8e10b3638 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:13.822938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-18319a1c-04c6-400e-bd4b-0bf8e10b3638 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:13.828970 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18319a1c-04c6-400e-bd4b-0bf8e10b3638 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:13.829413 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18319a1c-04c6-400e-bd4b-0bf8e10b3638 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:13.830171 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18319a1c-04c6-400e-bd4b-0bf8e10b3638 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:13.830634 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-18319a1c-04c6-400e-bd4b-0bf8e10b3638 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:13.958655 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-18319a1c-04c6-400e-bd4b-0bf8e10b3638 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:13.967526 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-18319a1c-04c6-400e-bd4b-0bf8e10b3638 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:13.969068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-18319a1c-04c6-400e-bd4b-0bf8e10b3638 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:13.970214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-18319a1c-04c6-400e-bd4b-0bf8e10b3638 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1723 microversion: 2.20 time: 0.151879 Jul 05 10:22:13.970772 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 466/906] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:13 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:14.989455 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-12b4ab78-577a-4285-952b-d4fbb8c361b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:14.990173 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-12b4ab78-577a-4285-952b-d4fbb8c361b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:14.997420 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-12b4ab78-577a-4285-952b-d4fbb8c361b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:14.997868 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-12b4ab78-577a-4285-952b-d4fbb8c361b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:14.998621 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-12b4ab78-577a-4285-952b-d4fbb8c361b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:14.999054 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-12b4ab78-577a-4285-952b-d4fbb8c361b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:15.134255 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-12b4ab78-577a-4285-952b-d4fbb8c361b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:15.142332 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-12b4ab78-577a-4285-952b-d4fbb8c361b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:15.143888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-12b4ab78-577a-4285-952b-d4fbb8c361b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:15.145138 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-12b4ab78-577a-4285-952b-d4fbb8c361b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1723 microversion: 2.20 time: 0.159893 Jul 05 10:22:15.145690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 441/907] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:14 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1723 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:16.166185 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-bbbefbe2-28d5-4d54-8896-f4b2d7927f40 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:16.166704 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-bbbefbe2-28d5-4d54-8896-f4b2d7927f40 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:16.173408 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bbbefbe2-28d5-4d54-8896-f4b2d7927f40 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:16.173867 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bbbefbe2-28d5-4d54-8896-f4b2d7927f40 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:16.174646 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bbbefbe2-28d5-4d54-8896-f4b2d7927f40 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:16.175086 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bbbefbe2-28d5-4d54-8896-f4b2d7927f40 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:16.389565 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-bbbefbe2-28d5-4d54-8896-f4b2d7927f40 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:16.397635 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-bbbefbe2-28d5-4d54-8896-f4b2d7927f40 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:16.399158 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-bbbefbe2-28d5-4d54-8896-f4b2d7927f40 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:16.400319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-bbbefbe2-28d5-4d54-8896-f4b2d7927f40 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 200 len: 1723 microversion: 2.20 time: 0.238602 Jul 05 10:22:16.401221 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 467/908] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:16 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 1723 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:16.783760 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-6ccbd626-6da9-4ae1-b67c-f65229fdebb9 req-75e0c6f6-1b99-42a8-b338-edb895194a7a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ff442846-3ded-48c4-8a48-df125f46d0d6", "name": "network-vif-deleted", "tag": "e6fb0593-584d-4308-86f5-4705f3dc20c8"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:22:16.792949 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-6ccbd626-6da9-4ae1-b67c-f65229fdebb9 req-75e0c6f6-1b99-42a8-b338-edb895194a7a service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:16.793536 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-6ccbd626-6da9-4ae1-b67c-f65229fdebb9 req-75e0c6f6-1b99-42a8-b338-edb895194a7a service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:16.805910 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-6ccbd626-6da9-4ae1-b67c-f65229fdebb9 req-75e0c6f6-1b99-42a8-b338-edb895194a7a service nova] Creating event network-vif-deleted:e6fb0593-584d-4308-86f5-4705f3dc20c8 for instance ff442846-3ded-48c4-8a48-df125f46d0d6 on hv-cinder-maste Jul 05 10:22:16.810846 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-6ccbd626-6da9-4ae1-b67c-f65229fdebb9 req-75e0c6f6-1b99-42a8-b338-edb895194a7a service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030657 Jul 05 10:22:16.811896 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 442/909] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:22:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:22:17.418782 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9c8dfc62-3323-4d6c-b45f-fd0b528b215a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:17.419385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9c8dfc62-3323-4d6c-b45f-fd0b528b215a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:17.428147 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9c8dfc62-3323-4d6c-b45f-fd0b528b215a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:17.428630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9c8dfc62-3323-4d6c-b45f-fd0b528b215a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:17.429410 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9c8dfc62-3323-4d6c-b45f-fd0b528b215a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:17.429847 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9c8dfc62-3323-4d6c-b45f-fd0b528b215a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:17.445487 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-9c8dfc62-3323-4d6c-b45f-fd0b528b215a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] HTTP exception thrown: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9c8dfc62-3323-4d6c-b45f-fd0b528b215a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Request method failure captured: Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 HTTP/1.1 Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA_xvpEELgE-eBUYHUhNsBeyr4iL6TX2JC5RBVSqxhiIRu0uvV5DpIroGwyHi1VHjPe6ULauFPG_08J8dk6bV7uqHLXSCeuk3phF15RwcRNPncEKU39B23DXNZT1IGuIE1R6RvECOKMp30Qx65b7eEKmzJ9mKE-_AEJ6raY_zH75ufY Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Openstack-Nova-Api-Version: 2.20 Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: f0c19d009f1b4ef09a40267bcf654107 Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:22:17.446987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:22:17.448750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f0c19d009f1b4ef09a40267bcf654107"}]}] Jul 05 10:22:17.448750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:22:17.448750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: f0c19d009f1b4ef09a40267bcf654107 Jul 05 10:22:17.448750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:22:17.448750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeShelveTestJSON-1813703734-project Jul 05 10:22:17.448750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:22:17.448750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:22:17.448750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: d4a608563189457dbcadf8d48acdd84a Jul 05 10:22:17.448750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1813703734-project Jul 05 10:22:17.448750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:22:17.448750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:22:17.448750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'ff442846-3ded-48c4-8a48-df125f46d0d6'} Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:22:17.449784 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:22:17.451239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:22:17.452647 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9c8dfc62-3323-4d6c-b45f-fd0b528b215a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Returning 404 to user: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:22:17.452647 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9c8dfc62-3323-4d6c-b45f-fd0b528b215a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 404 len: 111 microversion: 2.20 time: 0.033763 Jul 05 10:22:17.452647 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 468/910] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:17 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 05 10:22:17.458414 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5bd2eea4-fd4e-4bb7-b16a-6e9bae994a47 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '.version_select at 0x7feb895dbdc0>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:17.478043 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5bd2eea4-fd4e-4bb7-b16a-6e9bae994a47 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-691061805" status: 204 len: 0 microversion: 2.20 time: 0.023640 Jul 05 10:22:17.478628 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 443/911] 10.222.0.20 () {60 vars in 1258 bytes} [Tue Jul 5 10:22:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-691061805 => generated 0 bytes in 25 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 05 10:22:18.103479 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5da94214-91ac-4a79-84e4-b69119299325 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Action: 'create', calling method: .version_select at 0x7feb8deb6ee0>, body: {"keypair": {"name": "tempest-keypair-1899722997", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBH6rdX1+lqIQUKMFtjVSyT+gBlgX/mI79UOI0Q31oDZJnKjPH9YeLwV+qMOUrQPivr07Kz8A4lhQImlc7rOB2AbC8r5P1aQ8jRcE4qDDrFTiCV8WV5dMw6DDHCIkH2K2iQ=="}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:22:18.117095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-5da94214-91ac-4a79-84e4-b69119299325 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Getting quotas for project f0c19d009f1b4ef09a40267bcf654107. Resources: {'key_pairs'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:22:18.120922 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-5da94214-91ac-4a79-84e4-b69119299325 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Getting quotas for user d4a608563189457dbcadf8d48acdd84a and project f0c19d009f1b4ef09a40267bcf654107. Resources: {'key_pairs'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:22:18.142633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5da94214-91ac-4a79-84e4-b69119299325 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.043174 Jul 05 10:22:18.143212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 469/912] 10.222.0.20 () {62 vars in 1199 bytes} [Tue Jul 5 10:22:18 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 44 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 05 10:22:19.478070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1072901628", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "fddec1cf-8b7c-4e90-af09-12a66276106e"}], "security_groups": [{"name": "tempest-securitygroup--1631636744"}], "key_name": "tempest-keypair-1899722997", "user_data": "}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:22:19.580759 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.network.neutron [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] validate_networks() for [('fddec1cf-8b7c-4e90-af09-12a66276106e', None, None, None, None, None)] {{(pid=117812) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:22:19.962092 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Flavor limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:22:19.962511 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Image limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:22:19.962979 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Flavor pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:22:19.963385 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Image pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:22:19.963792 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:22:19.974382 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:19.975246 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:19.976211 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:19.976684 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:19.996006 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Getting quotas for project f0c19d009f1b4ef09a40267bcf654107. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:22:20.000943 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Getting quotas for user d4a608563189457dbcadf8d48acdd84a and project f0c19d009f1b4ef09a40267bcf654107. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:22:20.012651 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Going to run 1 instances... {{(pid=117812) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:22:20.015018 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117812) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:22:20.063391 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7bc744d2-838c-4009-9bb0-714d3103b3cd tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.590819 Jul 05 10:22:20.064244 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 444/913] 10.222.0.20 () {62 vars in 1187 bytes} [Tue Jul 5 10:22:19 2022] POST /compute/v2.1/servers => generated 402 bytes in 592 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 05 10:22:20.074917 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0b5975b2-2077-4c70-9054-5b84f852b359 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:20.076474 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0b5975b2-2077-4c70-9054-5b84f852b359 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:20.165377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0b5975b2-2077-4c70-9054-5b84f852b359 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:20.175088 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0b5975b2-2077-4c70-9054-5b84f852b359 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:20.176944 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0b5975b2-2077-4c70-9054-5b84f852b359 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:20.178052 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0b5975b2-2077-4c70-9054-5b84f852b359 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1282 microversion: 2.20 time: 0.107838 Jul 05 10:22:20.178598 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 470/914] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:20 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1282 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:21.197329 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fb29f26d-eb8b-42bf-b8d9-34ee9eaefb96 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:21.198215 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fb29f26d-eb8b-42bf-b8d9-34ee9eaefb96 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:21.206415 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb29f26d-eb8b-42bf-b8d9-34ee9eaefb96 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:21.207143 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb29f26d-eb8b-42bf-b8d9-34ee9eaefb96 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:21.208053 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb29f26d-eb8b-42bf-b8d9-34ee9eaefb96 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:21.208619 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fb29f26d-eb8b-42bf-b8d9-34ee9eaefb96 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:21.281160 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fb29f26d-eb8b-42bf-b8d9-34ee9eaefb96 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:21.290542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fb29f26d-eb8b-42bf-b8d9-34ee9eaefb96 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:21.292147 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fb29f26d-eb8b-42bf-b8d9-34ee9eaefb96 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:21.293365 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fb29f26d-eb8b-42bf-b8d9-34ee9eaefb96 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1352 microversion: 2.20 time: 0.100599 Jul 05 10:22:21.294045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 445/915] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:21 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1352 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:22.311403 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2a3d0eae-b5c2-46d1-bc7b-371bb0c650bb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:22.311948 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2a3d0eae-b5c2-46d1-bc7b-371bb0c650bb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:22.318485 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2a3d0eae-b5c2-46d1-bc7b-371bb0c650bb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:22.318924 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2a3d0eae-b5c2-46d1-bc7b-371bb0c650bb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:22.319673 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2a3d0eae-b5c2-46d1-bc7b-371bb0c650bb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:22.320104 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2a3d0eae-b5c2-46d1-bc7b-371bb0c650bb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:22.452199 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2a3d0eae-b5c2-46d1-bc7b-371bb0c650bb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:22.460636 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2a3d0eae-b5c2-46d1-bc7b-371bb0c650bb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:22.462142 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2a3d0eae-b5c2-46d1-bc7b-371bb0c650bb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:22.463249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2a3d0eae-b5c2-46d1-bc7b-371bb0c650bb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1408 microversion: 2.20 time: 0.155844 Jul 05 10:22:22.463795 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 471/916] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:22 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1408 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:23.481353 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-42102f99-8e2f-43bd-9ca7-a79e756dce1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:23.481960 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-42102f99-8e2f-43bd-9ca7-a79e756dce1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:23.489033 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-42102f99-8e2f-43bd-9ca7-a79e756dce1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:23.489484 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-42102f99-8e2f-43bd-9ca7-a79e756dce1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:23.490251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-42102f99-8e2f-43bd-9ca7-a79e756dce1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:23.490692 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-42102f99-8e2f-43bd-9ca7-a79e756dce1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:23.615583 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-42102f99-8e2f-43bd-9ca7-a79e756dce1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:23.623433 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-42102f99-8e2f-43bd-9ca7-a79e756dce1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:23.624964 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-42102f99-8e2f-43bd-9ca7-a79e756dce1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:23.626057 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-42102f99-8e2f-43bd-9ca7-a79e756dce1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1408 microversion: 2.20 time: 0.148638 Jul 05 10:22:23.626605 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 446/917] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:23 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1408 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:24.643598 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-474a8852-0836-4eb3-8780-c764d33d8141 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:24.644196 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-474a8852-0836-4eb3-8780-c764d33d8141 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:24.650598 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-474a8852-0836-4eb3-8780-c764d33d8141 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:24.651038 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-474a8852-0836-4eb3-8780-c764d33d8141 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:24.651783 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-474a8852-0836-4eb3-8780-c764d33d8141 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:24.652214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-474a8852-0836-4eb3-8780-c764d33d8141 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:24.774888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-474a8852-0836-4eb3-8780-c764d33d8141 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:24.782617 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-474a8852-0836-4eb3-8780-c764d33d8141 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:24.784652 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-474a8852-0836-4eb3-8780-c764d33d8141 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:24.785742 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-474a8852-0836-4eb3-8780-c764d33d8141 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1408 microversion: 2.20 time: 0.145940 Jul 05 10:22:24.786289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 472/918] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:24 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1408 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:25.521703 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-76b150b7-4a9b-43e6-bef7-5288a41b00a6 req-801c344a-7136-4e2a-adde-a28cfcba3f9a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1c90da36-1371-4861-9756-e275bf1f514a", "tag": "ceece817-3e21-46da-8f70-bbfe947c7cd6"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:22:25.546260 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-76b150b7-4a9b-43e6-bef7-5288a41b00a6 req-801c344a-7136-4e2a-adde-a28cfcba3f9a service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:25.547936 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-76b150b7-4a9b-43e6-bef7-5288a41b00a6 req-801c344a-7136-4e2a-adde-a28cfcba3f9a service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:25.561528 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-76b150b7-4a9b-43e6-bef7-5288a41b00a6 req-801c344a-7136-4e2a-adde-a28cfcba3f9a service nova] Creating event network-changed:ceece817-3e21-46da-8f70-bbfe947c7cd6 for instance 1c90da36-1371-4861-9756-e275bf1f514a on hv-cinder-maste Jul 05 10:22:25.567096 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-76b150b7-4a9b-43e6-bef7-5288a41b00a6 req-801c344a-7136-4e2a-adde-a28cfcba3f9a service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052174 Jul 05 10:22:25.567770 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 447/919] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:22:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:22:25.803881 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f92aa7ac-73c5-4904-bbad-861a2d00a1af tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:25.804516 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f92aa7ac-73c5-4904-bbad-861a2d00a1af tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:25.810835 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f92aa7ac-73c5-4904-bbad-861a2d00a1af tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:25.811369 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f92aa7ac-73c5-4904-bbad-861a2d00a1af tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:25.812195 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f92aa7ac-73c5-4904-bbad-861a2d00a1af tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:25.812647 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f92aa7ac-73c5-4904-bbad-861a2d00a1af tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:26.031645 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f92aa7ac-73c5-4904-bbad-861a2d00a1af tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:26.039684 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f92aa7ac-73c5-4904-bbad-861a2d00a1af tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:26.041239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f92aa7ac-73c5-4904-bbad-861a2d00a1af tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:26.042344 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f92aa7ac-73c5-4904-bbad-861a2d00a1af tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1408 microversion: 2.20 time: 0.243110 Jul 05 10:22:26.042891 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 473/920] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:25 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1408 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:27.061136 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-71f1e687-43d2-477a-ae03-f4205d5a8d47 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:27.061883 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-71f1e687-43d2-477a-ae03-f4205d5a8d47 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:27.068631 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-71f1e687-43d2-477a-ae03-f4205d5a8d47 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:27.069073 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-71f1e687-43d2-477a-ae03-f4205d5a8d47 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:27.069826 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-71f1e687-43d2-477a-ae03-f4205d5a8d47 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:27.070257 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-71f1e687-43d2-477a-ae03-f4205d5a8d47 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:27.200704 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-71f1e687-43d2-477a-ae03-f4205d5a8d47 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:27.209524 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-71f1e687-43d2-477a-ae03-f4205d5a8d47 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:27.211021 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-71f1e687-43d2-477a-ae03-f4205d5a8d47 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:27.212125 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-71f1e687-43d2-477a-ae03-f4205d5a8d47 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1408 microversion: 2.20 time: 0.155782 Jul 05 10:22:27.212711 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 448/921] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:27 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1408 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 05 10:22:28.231141 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0113fd9d-a105-4d09-a483-df58645c0372 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:28.232435 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0113fd9d-a105-4d09-a483-df58645c0372 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:28.239185 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0113fd9d-a105-4d09-a483-df58645c0372 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:28.239917 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0113fd9d-a105-4d09-a483-df58645c0372 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:28.241802 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0113fd9d-a105-4d09-a483-df58645c0372 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:28.242386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0113fd9d-a105-4d09-a483-df58645c0372 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:28.462057 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0113fd9d-a105-4d09-a483-df58645c0372 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:28.471422 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0113fd9d-a105-4d09-a483-df58645c0372 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:28.473437 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0113fd9d-a105-4d09-a483-df58645c0372 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:28.475687 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0113fd9d-a105-4d09-a483-df58645c0372 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1408 microversion: 2.20 time: 0.248885 Jul 05 10:22:28.476523 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 474/922] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:28 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1408 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:29.494188 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-df71cc05-8e4e-4103-9299-18e9d5aae71b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:29.494800 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-df71cc05-8e4e-4103-9299-18e9d5aae71b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:29.501627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-df71cc05-8e4e-4103-9299-18e9d5aae71b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:29.502255 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-df71cc05-8e4e-4103-9299-18e9d5aae71b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:29.503197 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-df71cc05-8e4e-4103-9299-18e9d5aae71b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:29.503636 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-df71cc05-8e4e-4103-9299-18e9d5aae71b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:29.639297 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-df71cc05-8e4e-4103-9299-18e9d5aae71b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:29.649119 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-df71cc05-8e4e-4103-9299-18e9d5aae71b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:29.650832 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-df71cc05-8e4e-4103-9299-18e9d5aae71b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:29.651949 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-df71cc05-8e4e-4103-9299-18e9d5aae71b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1408 microversion: 2.20 time: 0.161683 Jul 05 10:22:29.652537 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 449/923] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:29 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1408 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:30.671494 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-95e72837-17aa-4bd1-9731-0eafe132e5eb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:30.672956 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-95e72837-17aa-4bd1-9731-0eafe132e5eb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:30.680287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=6 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:30.682305 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-95e72837-17aa-4bd1-9731-0eafe132e5eb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:30.683229 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-95e72837-17aa-4bd1-9731-0eafe132e5eb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:30.684316 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-95e72837-17aa-4bd1-9731-0eafe132e5eb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:30.684995 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-95e72837-17aa-4bd1-9731-0eafe132e5eb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:30.844662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-95e72837-17aa-4bd1-9731-0eafe132e5eb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:30.853476 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-95e72837-17aa-4bd1-9731-0eafe132e5eb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:30.855486 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-95e72837-17aa-4bd1-9731-0eafe132e5eb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:30.856457 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-95e72837-17aa-4bd1-9731-0eafe132e5eb tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1408 microversion: 2.20 time: 0.189990 Jul 05 10:22:30.857005 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 475/924] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:30 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1408 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:31.875620 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e706c0e5-d113-411d-915c-769d74554819 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:31.877633 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e706c0e5-d113-411d-915c-769d74554819 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:31.883578 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:31.884733 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=6 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:31.887697 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e706c0e5-d113-411d-915c-769d74554819 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:31.889156 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e706c0e5-d113-411d-915c-769d74554819 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:31.890491 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e706c0e5-d113-411d-915c-769d74554819 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:31.892466 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e706c0e5-d113-411d-915c-769d74554819 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:32.098136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-ac7309bf-33ab-43ea-8413-23dcaa257c8b req-b897279b-7497-4c49-8052-67d642ff4f80 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c90da36-1371-4861-9756-e275bf1f514a", "name": "network-vif-plugged", "status": "completed", "tag": "ceece817-3e21-46da-8f70-bbfe947c7cd6"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:22:32.113296 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e706c0e5-d113-411d-915c-769d74554819 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:32.119725 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-ac7309bf-33ab-43ea-8413-23dcaa257c8b req-b897279b-7497-4c49-8052-67d642ff4f80 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:32.121797 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-ac7309bf-33ab-43ea-8413-23dcaa257c8b req-b897279b-7497-4c49-8052-67d642ff4f80 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:32.124020 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e706c0e5-d113-411d-915c-769d74554819 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:32.125977 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e706c0e5-d113-411d-915c-769d74554819 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:32.127845 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e706c0e5-d113-411d-915c-769d74554819 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1408 microversion: 2.20 time: 0.256826 Jul 05 10:22:32.129117 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 450/925] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:31 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1408 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:32.143104 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-ac7309bf-33ab-43ea-8413-23dcaa257c8b req-b897279b-7497-4c49-8052-67d642ff4f80 service nova] Creating event network-vif-plugged:ceece817-3e21-46da-8f70-bbfe947c7cd6 for instance 1c90da36-1371-4861-9756-e275bf1f514a on hv-cinder-maste Jul 05 10:22:32.159595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-ac7309bf-33ab-43ea-8413-23dcaa257c8b req-b897279b-7497-4c49-8052-67d642ff4f80 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068790 Jul 05 10:22:32.161365 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 476/926] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:22:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:22:33.146696 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-72890231-dcb4-4a3c-ae81-ff5b78163209 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:33.147286 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-72890231-dcb4-4a3c-ae81-ff5b78163209 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:33.156753 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72890231-dcb4-4a3c-ae81-ff5b78163209 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:33.157218 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72890231-dcb4-4a3c-ae81-ff5b78163209 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:33.157987 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72890231-dcb4-4a3c-ae81-ff5b78163209 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:33.165315 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-72890231-dcb4-4a3c-ae81-ff5b78163209 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:33.347371 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-72890231-dcb4-4a3c-ae81-ff5b78163209 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:33.356814 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-72890231-dcb4-4a3c-ae81-ff5b78163209 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:33.359285 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-72890231-dcb4-4a3c-ae81-ff5b78163209 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:33.360741 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-72890231-dcb4-4a3c-ae81-ff5b78163209 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1408 microversion: 2.20 time: 0.217918 Jul 05 10:22:33.364336 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 451/927] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:33 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1408 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:34.383092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-daab5ee2-868c-4528-a180-c30ea603f7f0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:34.384084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-daab5ee2-868c-4528-a180-c30ea603f7f0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:34.392815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-daab5ee2-868c-4528-a180-c30ea603f7f0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:34.393460 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-daab5ee2-868c-4528-a180-c30ea603f7f0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:34.394767 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-daab5ee2-868c-4528-a180-c30ea603f7f0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:34.395353 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-daab5ee2-868c-4528-a180-c30ea603f7f0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:34.638725 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-daab5ee2-868c-4528-a180-c30ea603f7f0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:34.647032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-daab5ee2-868c-4528-a180-c30ea603f7f0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:34.648794 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-daab5ee2-868c-4528-a180-c30ea603f7f0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:22:34.649951 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-daab5ee2-868c-4528-a180-c30ea603f7f0 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1601 microversion: 2.20 time: 0.271603 Jul 05 10:22:34.650507 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 477/928] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:22:34 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1601 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:22:34.667558 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-636c137c-8f68-40bb-bd3f-20f092e087ff tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:22:34.668122 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-636c137c-8f68-40bb-bd3f-20f092e087ff tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:22:34.675204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-636c137c-8f68-40bb-bd3f-20f092e087ff tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:34.675655 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-636c137c-8f68-40bb-bd3f-20f092e087ff tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:34.676449 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-636c137c-8f68-40bb-bd3f-20f092e087ff tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:34.676893 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-636c137c-8f68-40bb-bd3f-20f092e087ff tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:35.077173 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-636c137c-8f68-40bb-bd3f-20f092e087ff tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.413172 Jul 05 10:22:35.077962 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 452/929] 10.222.0.20 () {60 vars in 1315 bytes} [Tue Jul 5 10:22:34 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/os-interface => generated 285 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:22:36.839405 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-9a510086-99a5-4aa9-bc13-b7df1e8a165d req-b8aea25f-bdda-4f19-b80f-0f7001c9241b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1c90da36-1371-4861-9756-e275bf1f514a", "tag": "ceece817-3e21-46da-8f70-bbfe947c7cd6"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:22:36.848913 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-9a510086-99a5-4aa9-bc13-b7df1e8a165d req-b8aea25f-bdda-4f19-b80f-0f7001c9241b service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:22:36.849595 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-9a510086-99a5-4aa9-bc13-b7df1e8a165d req-b8aea25f-bdda-4f19-b80f-0f7001c9241b service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:22:36.865341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-9a510086-99a5-4aa9-bc13-b7df1e8a165d req-b8aea25f-bdda-4f19-b80f-0f7001c9241b service nova] Creating event network-changed:ceece817-3e21-46da-8f70-bbfe947c7cd6 for instance 1c90da36-1371-4861-9756-e275bf1f514a on hv-cinder-maste Jul 05 10:22:36.870871 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-9a510086-99a5-4aa9-bc13-b7df1e8a165d req-b8aea25f-bdda-4f19-b80f-0f7001c9241b service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035098 Jul 05 10:22:36.874550 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 478/930] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:22:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:22:39.839621 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-aa15b3f3-32e4-4fea-8ae5-c5e21007e465 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Action: 'create', calling method: .version_select at 0x7feb895db9d0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1298886845", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG7C8Xzun8CunZV296rT47RUqJObjc/2g6GC3Fk/6WPVExImLoavlPxuzw8tGssitsXR/bVJJ+43nD9ebZ+4hX5TVxQH23ZjTAzxbY59/Ss+OzNc/GAgCn3dsoEc8rgS+w=="}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:22:39.844171 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=52,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:22:39.859592 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-aa15b3f3-32e4-4fea-8ae5-c5e21007e465 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Getting quotas for project 86eac45249ad4a5e9513fcc091bcf42e. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:22:39.864031 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-aa15b3f3-32e4-4fea-8ae5-c5e21007e465 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Getting quotas for user c9203890ad7649f897b13ac7bdf6be3b and project 86eac45249ad4a5e9513fcc091bcf42e. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:22:39.886825 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-aa15b3f3-32e4-4fea-8ae5-c5e21007e465 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.250089 Jul 05 10:22:39.887444 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 453/931] 10.222.0.20 () {60 vars in 1158 bytes} [Tue Jul 5 10:22:39 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:23:03.371631 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=45,nova_api:INSERT=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:03.372359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=78 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:03.376587 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1593409159", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1298886845", "security_groups": [{"name": "tempest-secgroup-smoke-1344850936"}], "block_device_mapping_v2": [{"uuid": "da4c16ca-15da-4cce-a606-40d05ba2cc45", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "5a6b68d3-f3f2-400d-a2f3-b1cc33d83760"}]}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:23:03.486583 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.network.neutron [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] validate_networks() for [('5a6b68d3-f3f2-400d-a2f3-b1cc33d83760', None, None, None, None, None)] {{(pid=117813) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:23:03.982215 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Flavor limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:23:03.982631 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Image limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:23:03.983120 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Flavor pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:23:03.983516 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Image pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:23:03.983952 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:23:03.992653 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:03.993098 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:03.993909 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:03.994341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:04.010960 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Getting quotas for project 86eac45249ad4a5e9513fcc091bcf42e. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:23:04.014768 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Getting quotas for user c9203890ad7649f897b13ac7bdf6be3b and project 86eac45249ad4a5e9513fcc091bcf42e. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:23:04.069812 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Going to run 1 instances... {{(pid=117813) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:23:04.071958 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='da4c16ca-15da-4cce-a606-40d05ba2cc45',volume_size=None,volume_type=None)] {{(pid=117813) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:23:04.196697 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-6e6e8ae6-4eee-4ca2-a733-dbdb5caec3a5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.826473 Jul 05 10:23:04.197344 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 479/932] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:23:03 2022] POST /compute/v2.1/servers => generated 406 bytes in 828 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:23:04.204654 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=87,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:04.205495 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=4 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:04.206245 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=10,nova_api:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:04.211485 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-9da60969-b0c3-4e4d-9d58-626030eac010 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:04.212084 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-9da60969-b0c3-4e4d-9d58-626030eac010 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:04.276607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-9da60969-b0c3-4e4d-9d58-626030eac010 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:04.283464 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-9da60969-b0c3-4e4d-9d58-626030eac010 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1095 microversion: 2.1 time: 0.079980 Jul 05 10:23:04.284013 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 454/933] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:04 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1095 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:05.300446 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2da3711b-e131-4fa9-af76-6007f4a7f696 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:05.301064 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2da3711b-e131-4fa9-af76-6007f4a7f696 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:05.308105 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2da3711b-e131-4fa9-af76-6007f4a7f696 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:05.308588 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2da3711b-e131-4fa9-af76-6007f4a7f696 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:05.309352 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2da3711b-e131-4fa9-af76-6007f4a7f696 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:05.309808 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2da3711b-e131-4fa9-af76-6007f4a7f696 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:05.388704 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2da3711b-e131-4fa9-af76-6007f4a7f696 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:05.396583 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2da3711b-e131-4fa9-af76-6007f4a7f696 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1211 microversion: 2.1 time: 0.100370 Jul 05 10:23:05.397148 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 480/934] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:05 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1211 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:06.414010 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ba84331c-5b50-445d-90ac-c7efc5d8da9a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:06.414486 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ba84331c-5b50-445d-90ac-c7efc5d8da9a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:06.421865 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ba84331c-5b50-445d-90ac-c7efc5d8da9a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:06.422407 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ba84331c-5b50-445d-90ac-c7efc5d8da9a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:06.423504 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ba84331c-5b50-445d-90ac-c7efc5d8da9a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:06.424115 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ba84331c-5b50-445d-90ac-c7efc5d8da9a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:06.607844 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-bade944d-cde4-40f4-9c9a-10942d08f75d req-69a87e08-b0e2-4bd4-a919-eafe9839d608 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1eeacca5-4921-455f-80e9-acee90ae762f", "tag": "265f0d38-e515-44a1-95c5-3695adc2b78e"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:23:06.623765 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-bade944d-cde4-40f4-9c9a-10942d08f75d req-69a87e08-b0e2-4bd4-a919-eafe9839d608 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:06.624675 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-bade944d-cde4-40f4-9c9a-10942d08f75d req-69a87e08-b0e2-4bd4-a919-eafe9839d608 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:06.636986 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-bade944d-cde4-40f4-9c9a-10942d08f75d req-69a87e08-b0e2-4bd4-a919-eafe9839d608 service nova] Creating event network-changed:265f0d38-e515-44a1-95c5-3695adc2b78e for instance 1eeacca5-4921-455f-80e9-acee90ae762f on hv-cinder-maste Jul 05 10:23:06.645638 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ba84331c-5b50-445d-90ac-c7efc5d8da9a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:06.650212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-bade944d-cde4-40f4-9c9a-10942d08f75d req-69a87e08-b0e2-4bd4-a919-eafe9839d608 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047484 Jul 05 10:23:06.651041 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 481/935] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:23:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:23:06.653928 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ba84331c-5b50-445d-90ac-c7efc5d8da9a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1267 microversion: 2.1 time: 0.244504 Jul 05 10:23:06.654527 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 455/936] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:06 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1267 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:07.672001 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-76122954-e0f3-4156-8af6-f763b8561724 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:07.672710 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-76122954-e0f3-4156-8af6-f763b8561724 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:07.679278 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-76122954-e0f3-4156-8af6-f763b8561724 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:07.679722 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-76122954-e0f3-4156-8af6-f763b8561724 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:07.680502 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-76122954-e0f3-4156-8af6-f763b8561724 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:07.680943 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-76122954-e0f3-4156-8af6-f763b8561724 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:07.811914 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-76122954-e0f3-4156-8af6-f763b8561724 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:07.819878 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-76122954-e0f3-4156-8af6-f763b8561724 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1267 microversion: 2.1 time: 0.152408 Jul 05 10:23:07.820457 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 482/937] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:07 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:08.836746 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f9e66e31-1a97-41bf-8420-67c4be5b9c1d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:08.837317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f9e66e31-1a97-41bf-8420-67c4be5b9c1d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:08.843901 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f9e66e31-1a97-41bf-8420-67c4be5b9c1d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:08.844357 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f9e66e31-1a97-41bf-8420-67c4be5b9c1d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:08.845151 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f9e66e31-1a97-41bf-8420-67c4be5b9c1d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:08.845592 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f9e66e31-1a97-41bf-8420-67c4be5b9c1d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:08.976704 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f9e66e31-1a97-41bf-8420-67c4be5b9c1d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:08.984073 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f9e66e31-1a97-41bf-8420-67c4be5b9c1d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1267 microversion: 2.1 time: 0.151544 Jul 05 10:23:08.984670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 456/938] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:08 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:10.000659 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f89ad55e-393f-4604-b2fa-b27d01ca4b61 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:10.001263 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f89ad55e-393f-4604-b2fa-b27d01ca4b61 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:10.008189 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f89ad55e-393f-4604-b2fa-b27d01ca4b61 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:10.008664 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f89ad55e-393f-4604-b2fa-b27d01ca4b61 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:10.009414 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f89ad55e-393f-4604-b2fa-b27d01ca4b61 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:10.009844 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f89ad55e-393f-4604-b2fa-b27d01ca4b61 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:10.138071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f89ad55e-393f-4604-b2fa-b27d01ca4b61 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:10.145526 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f89ad55e-393f-4604-b2fa-b27d01ca4b61 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1267 microversion: 2.1 time: 0.148783 Jul 05 10:23:10.146083 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 483/939] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:09 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1267 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:11.161789 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e9136ad1-597c-4361-a6a5-fab56dcbd790 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:11.162398 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e9136ad1-597c-4361-a6a5-fab56dcbd790 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:11.168938 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e9136ad1-597c-4361-a6a5-fab56dcbd790 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:11.169394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e9136ad1-597c-4361-a6a5-fab56dcbd790 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:11.170158 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e9136ad1-597c-4361-a6a5-fab56dcbd790 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:11.170597 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e9136ad1-597c-4361-a6a5-fab56dcbd790 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:11.302157 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e9136ad1-597c-4361-a6a5-fab56dcbd790 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:11.309572 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e9136ad1-597c-4361-a6a5-fab56dcbd790 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1267 microversion: 2.1 time: 0.151631 Jul 05 10:23:11.310140 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 457/940] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:11 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:12.326788 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6caf2754-3af4-4ca6-abdb-a8a6e89d0231 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:12.327400 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-6caf2754-3af4-4ca6-abdb-a8a6e89d0231 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:12.334725 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6caf2754-3af4-4ca6-abdb-a8a6e89d0231 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:12.335182 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6caf2754-3af4-4ca6-abdb-a8a6e89d0231 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:12.335940 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6caf2754-3af4-4ca6-abdb-a8a6e89d0231 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:12.336378 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6caf2754-3af4-4ca6-abdb-a8a6e89d0231 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:12.464912 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6caf2754-3af4-4ca6-abdb-a8a6e89d0231 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:12.472171 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-6caf2754-3af4-4ca6-abdb-a8a6e89d0231 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1267 microversion: 2.1 time: 0.149265 Jul 05 10:23:12.472770 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 484/941] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:12 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1267 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:13.488575 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3eaa0c14-fd3b-4a1f-b02b-5acae03a9bc4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:13.489211 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3eaa0c14-fd3b-4a1f-b02b-5acae03a9bc4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:13.495556 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3eaa0c14-fd3b-4a1f-b02b-5acae03a9bc4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:13.496005 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3eaa0c14-fd3b-4a1f-b02b-5acae03a9bc4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:13.496791 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3eaa0c14-fd3b-4a1f-b02b-5acae03a9bc4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:13.497226 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3eaa0c14-fd3b-4a1f-b02b-5acae03a9bc4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:13.628813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-3eaa0c14-fd3b-4a1f-b02b-5acae03a9bc4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:13.636122 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3eaa0c14-fd3b-4a1f-b02b-5acae03a9bc4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1267 microversion: 2.1 time: 0.151402 Jul 05 10:23:13.636718 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 458/942] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:13 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:14.653144 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b32d8ddb-6f04-4015-b3ab-5ded4692ef17 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:14.653733 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b32d8ddb-6f04-4015-b3ab-5ded4692ef17 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:14.656214 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:14.657068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=6 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:14.663416 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b32d8ddb-6f04-4015-b3ab-5ded4692ef17 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:14.663864 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b32d8ddb-6f04-4015-b3ab-5ded4692ef17 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:14.664661 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b32d8ddb-6f04-4015-b3ab-5ded4692ef17 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:14.665095 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b32d8ddb-6f04-4015-b3ab-5ded4692ef17 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:14.794000 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b32d8ddb-6f04-4015-b3ab-5ded4692ef17 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:14.802104 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b32d8ddb-6f04-4015-b3ab-5ded4692ef17 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1267 microversion: 2.1 time: 0.152704 Jul 05 10:23:14.802674 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 485/943] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:14 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:15.818650 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e28a33c3-e299-4640-8348-b3e01eb7e7ef tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:15.819250 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e28a33c3-e299-4640-8348-b3e01eb7e7ef tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:15.821143 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:15.826657 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e28a33c3-e299-4640-8348-b3e01eb7e7ef tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:15.827104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e28a33c3-e299-4640-8348-b3e01eb7e7ef tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:15.827858 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e28a33c3-e299-4640-8348-b3e01eb7e7ef tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:15.828290 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e28a33c3-e299-4640-8348-b3e01eb7e7ef tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:15.962558 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e28a33c3-e299-4640-8348-b3e01eb7e7ef tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:15.970127 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e28a33c3-e299-4640-8348-b3e01eb7e7ef tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1267 microversion: 2.1 time: 0.155245 Jul 05 10:23:15.970699 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 459/944] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:15 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1267 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:16.227771 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-964d6c00-ccc8-4ac5-9ef7-5910e41a1aa4 req-c1afaccf-b2c7-4114-a469-4417f212c6dd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1eeacca5-4921-455f-80e9-acee90ae762f", "name": "network-vif-plugged", "status": "completed", "tag": "265f0d38-e515-44a1-95c5-3695adc2b78e"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:23:16.235783 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-964d6c00-ccc8-4ac5-9ef7-5910e41a1aa4 req-c1afaccf-b2c7-4114-a469-4417f212c6dd service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:16.236226 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-964d6c00-ccc8-4ac5-9ef7-5910e41a1aa4 req-c1afaccf-b2c7-4114-a469-4417f212c6dd service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:16.247140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-964d6c00-ccc8-4ac5-9ef7-5910e41a1aa4 req-c1afaccf-b2c7-4114-a469-4417f212c6dd service nova] Creating event network-vif-plugged:265f0d38-e515-44a1-95c5-3695adc2b78e for instance 1eeacca5-4921-455f-80e9-acee90ae762f on hv-cinder-maste Jul 05 10:23:16.252083 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-964d6c00-ccc8-4ac5-9ef7-5910e41a1aa4 req-c1afaccf-b2c7-4114-a469-4417f212c6dd service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093900 Jul 05 10:23:16.252700 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 486/945] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:23:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:23:16.986975 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5d3f1e1a-0010-49c9-8702-7dbbe4a9a627 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:16.988187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5d3f1e1a-0010-49c9-8702-7dbbe4a9a627 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:16.994735 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5d3f1e1a-0010-49c9-8702-7dbbe4a9a627 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:16.995187 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5d3f1e1a-0010-49c9-8702-7dbbe4a9a627 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:16.995955 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5d3f1e1a-0010-49c9-8702-7dbbe4a9a627 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:16.996418 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5d3f1e1a-0010-49c9-8702-7dbbe4a9a627 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:17.124951 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5d3f1e1a-0010-49c9-8702-7dbbe4a9a627 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:17.132639 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5d3f1e1a-0010-49c9-8702-7dbbe4a9a627 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1267 microversion: 2.1 time: 0.149517 Jul 05 10:23:17.133206 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 460/946] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:16 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1267 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:18.150294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c51e3892-00d1-4e17-8bc4-744e18698754 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:18.150980 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c51e3892-00d1-4e17-8bc4-744e18698754 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:18.157154 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c51e3892-00d1-4e17-8bc4-744e18698754 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:18.157599 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c51e3892-00d1-4e17-8bc4-744e18698754 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:18.158348 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c51e3892-00d1-4e17-8bc4-744e18698754 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:18.158778 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c51e3892-00d1-4e17-8bc4-744e18698754 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:18.297049 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c51e3892-00d1-4e17-8bc4-744e18698754 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:18.305212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c51e3892-00d1-4e17-8bc4-744e18698754 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1267 microversion: 2.1 time: 0.159726 Jul 05 10:23:18.305783 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 487/947] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:18 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:19.322209 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-23413cd6-74e0-4d57-8b47-d49f5c9a4946 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:19.322793 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-23413cd6-74e0-4d57-8b47-d49f5c9a4946 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:19.329243 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-23413cd6-74e0-4d57-8b47-d49f5c9a4946 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:19.329684 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-23413cd6-74e0-4d57-8b47-d49f5c9a4946 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:19.331394 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-23413cd6-74e0-4d57-8b47-d49f5c9a4946 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:19.332117 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-23413cd6-74e0-4d57-8b47-d49f5c9a4946 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:19.459301 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-23413cd6-74e0-4d57-8b47-d49f5c9a4946 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:19.466675 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-23413cd6-74e0-4d57-8b47-d49f5c9a4946 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1453 microversion: 2.1 time: 0.148386 Jul 05 10:23:19.467228 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 461/948] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:19 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1453 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:19.482402 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d7ee65c3-4b6e-4aa7-89a9-a87b817513a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:19.483030 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d7ee65c3-4b6e-4aa7-89a9-a87b817513a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:19.489945 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d7ee65c3-4b6e-4aa7-89a9-a87b817513a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:19.490400 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d7ee65c3-4b6e-4aa7-89a9-a87b817513a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:19.491165 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d7ee65c3-4b6e-4aa7-89a9-a87b817513a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:19.491603 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d7ee65c3-4b6e-4aa7-89a9-a87b817513a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:19.620988 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d7ee65c3-4b6e-4aa7-89a9-a87b817513a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:19.629011 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d7ee65c3-4b6e-4aa7-89a9-a87b817513a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1453 microversion: 2.1 time: 0.150423 Jul 05 10:23:19.629573 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 488/949] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:23:19 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1453 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:23:21.211569 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-6c26df31-4cdf-4d74-bb60-c8933d71d198 req-03b83d40-cab8-425d-ac0f-7b353ea46d85 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1eeacca5-4921-455f-80e9-acee90ae762f", "tag": "265f0d38-e515-44a1-95c5-3695adc2b78e"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:23:21.219930 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-6c26df31-4cdf-4d74-bb60-c8933d71d198 req-03b83d40-cab8-425d-ac0f-7b353ea46d85 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:21.220375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-6c26df31-4cdf-4d74-bb60-c8933d71d198 req-03b83d40-cab8-425d-ac0f-7b353ea46d85 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:21.231522 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-6c26df31-4cdf-4d74-bb60-c8933d71d198 req-03b83d40-cab8-425d-ac0f-7b353ea46d85 service nova] Creating event network-changed:265f0d38-e515-44a1-95c5-3695adc2b78e for instance 1eeacca5-4921-455f-80e9-acee90ae762f on hv-cinder-maste Jul 05 10:23:21.237110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-6c26df31-4cdf-4d74-bb60-c8933d71d198 req-03b83d40-cab8-425d-ac0f-7b353ea46d85 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.028654 Jul 05 10:23:21.237711 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 462/950] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:23:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:23:52.220368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:52.221115 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=48 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:52.323846 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-66e8ee8d-6b00-41d0-bf8b-93645f6dbfe3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:52.324355 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-66e8ee8d-6b00-41d0-bf8b-93645f6dbfe3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:52.330276 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-66e8ee8d-6b00-41d0-bf8b-93645f6dbfe3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:52.330724 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-66e8ee8d-6b00-41d0-bf8b-93645f6dbfe3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:52.331478 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-66e8ee8d-6b00-41d0-bf8b-93645f6dbfe3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:52.331917 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-66e8ee8d-6b00-41d0-bf8b-93645f6dbfe3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:52.356108 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-66e8ee8d-6b00-41d0-bf8b-93645f6dbfe3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/ips" status: 200 len: 152 microversion: 2.20 time: 0.140454 Jul 05 10:23:52.356693 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 489/951] 10.222.0.20 () {60 vars in 1288 bytes} [Tue Jul 5 10:23:52 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/ips => generated 152 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:23:54.131082 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=20 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:54.131878 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=44 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:23:54.136229 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f96e343f-888e-4936-a378-15e8c0e10800 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:23:54.136862 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f96e343f-888e-4936-a378-15e8c0e10800 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:54.144226 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f96e343f-888e-4936-a378-15e8c0e10800 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:54.144703 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f96e343f-888e-4936-a378-15e8c0e10800 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:54.145464 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f96e343f-888e-4936-a378-15e8c0e10800 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:54.145900 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f96e343f-888e-4936-a378-15e8c0e10800 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:54.167424 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-f96e343f-888e-4936-a378-15e8c0e10800 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lazy-loading 'flavor' on Instance uuid 1c90da36-1371-4861-9756-e275bf1f514a {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:23:54.417107 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f96e343f-888e-4936-a378-15e8c0e10800 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "POST /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/action" status: 202 len: 0 microversion: 2.20 time: 0.287311 Jul 05 10:23:54.417735 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 463/952] 10.222.0.20 () {62 vars in 1318 bytes} [Tue Jul 5 10:23:54 2022] POST /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/action => generated 0 bytes in 289 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:23:54.425775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6348cf97-1bdb-4f41-b1fc-49f2148ef691 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:54.426515 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-6348cf97-1bdb-4f41-b1fc-49f2148ef691 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:54.433192 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6348cf97-1bdb-4f41-b1fc-49f2148ef691 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:54.433640 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6348cf97-1bdb-4f41-b1fc-49f2148ef691 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:54.434908 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6348cf97-1bdb-4f41-b1fc-49f2148ef691 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:54.435342 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6348cf97-1bdb-4f41-b1fc-49f2148ef691 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:54.567618 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6348cf97-1bdb-4f41-b1fc-49f2148ef691 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:54.575703 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6348cf97-1bdb-4f41-b1fc-49f2148ef691 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:54.577242 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6348cf97-1bdb-4f41-b1fc-49f2148ef691 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:54.578395 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-6348cf97-1bdb-4f41-b1fc-49f2148ef691 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1725 microversion: 2.20 time: 0.156537 Jul 05 10:23:54.578954 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 490/953] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:23:54 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1725 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:23:55.598738 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-579bde08-bdd1-400d-bcc4-6b2bbaf40ea6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:55.599439 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-579bde08-bdd1-400d-bcc4-6b2bbaf40ea6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:55.605781 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-579bde08-bdd1-400d-bcc4-6b2bbaf40ea6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:55.606238 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-579bde08-bdd1-400d-bcc4-6b2bbaf40ea6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:55.607007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-579bde08-bdd1-400d-bcc4-6b2bbaf40ea6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:55.607453 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-579bde08-bdd1-400d-bcc4-6b2bbaf40ea6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:55.735125 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-579bde08-bdd1-400d-bcc4-6b2bbaf40ea6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:55.742932 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-579bde08-bdd1-400d-bcc4-6b2bbaf40ea6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:55.744459 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-579bde08-bdd1-400d-bcc4-6b2bbaf40ea6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:55.745628 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-579bde08-bdd1-400d-bcc4-6b2bbaf40ea6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1725 microversion: 2.20 time: 0.151624 Jul 05 10:23:55.746683 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 464/954] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:23:55 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:23:56.765151 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6a79faf5-3f36-4c85-97cc-261dd5c5ce09 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:56.765757 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-6a79faf5-3f36-4c85-97cc-261dd5c5ce09 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:56.771772 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6a79faf5-3f36-4c85-97cc-261dd5c5ce09 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:56.772216 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6a79faf5-3f36-4c85-97cc-261dd5c5ce09 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:56.773010 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6a79faf5-3f36-4c85-97cc-261dd5c5ce09 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:56.773446 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6a79faf5-3f36-4c85-97cc-261dd5c5ce09 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:56.904779 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6a79faf5-3f36-4c85-97cc-261dd5c5ce09 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:56.912581 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6a79faf5-3f36-4c85-97cc-261dd5c5ce09 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:56.914076 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6a79faf5-3f36-4c85-97cc-261dd5c5ce09 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:56.915216 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-6a79faf5-3f36-4c85-97cc-261dd5c5ce09 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1725 microversion: 2.20 time: 0.154723 Jul 05 10:23:56.915779 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 491/955] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:23:56 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:23:57.933345 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8c9baa5a-ce73-4026-a0d1-8e79057a4115 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:57.933956 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-8c9baa5a-ce73-4026-a0d1-8e79057a4115 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:57.940522 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8c9baa5a-ce73-4026-a0d1-8e79057a4115 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:57.940983 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8c9baa5a-ce73-4026-a0d1-8e79057a4115 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:57.941746 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8c9baa5a-ce73-4026-a0d1-8e79057a4115 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:57.942190 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8c9baa5a-ce73-4026-a0d1-8e79057a4115 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:58.068191 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8c9baa5a-ce73-4026-a0d1-8e79057a4115 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:58.076285 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8c9baa5a-ce73-4026-a0d1-8e79057a4115 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:58.077823 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8c9baa5a-ce73-4026-a0d1-8e79057a4115 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:58.078976 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-8c9baa5a-ce73-4026-a0d1-8e79057a4115 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1725 microversion: 2.20 time: 0.149576 Jul 05 10:23:58.079526 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 465/956] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:23:57 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1725 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:23:59.097512 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f47e1c09-a7bc-4908-8ea5-b9ea2a1e1135 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:23:59.098116 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f47e1c09-a7bc-4908-8ea5-b9ea2a1e1135 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:23:59.105166 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f47e1c09-a7bc-4908-8ea5-b9ea2a1e1135 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:59.105609 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f47e1c09-a7bc-4908-8ea5-b9ea2a1e1135 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:59.106365 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f47e1c09-a7bc-4908-8ea5-b9ea2a1e1135 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:23:59.106801 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f47e1c09-a7bc-4908-8ea5-b9ea2a1e1135 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:23:59.237165 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f47e1c09-a7bc-4908-8ea5-b9ea2a1e1135 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:59.245018 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f47e1c09-a7bc-4908-8ea5-b9ea2a1e1135 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:59.247080 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f47e1c09-a7bc-4908-8ea5-b9ea2a1e1135 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:23:59.248237 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f47e1c09-a7bc-4908-8ea5-b9ea2a1e1135 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1725 microversion: 2.20 time: 0.154632 Jul 05 10:23:59.248851 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 492/957] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:23:59 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:00.266452 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d91b1cae-b9ac-432e-ad1c-afae4f6d2c6b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:00.266980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d91b1cae-b9ac-432e-ad1c-afae4f6d2c6b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:00.273439 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d91b1cae-b9ac-432e-ad1c-afae4f6d2c6b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:00.273899 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d91b1cae-b9ac-432e-ad1c-afae4f6d2c6b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:00.274664 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d91b1cae-b9ac-432e-ad1c-afae4f6d2c6b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:00.275108 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d91b1cae-b9ac-432e-ad1c-afae4f6d2c6b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:00.400551 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d91b1cae-b9ac-432e-ad1c-afae4f6d2c6b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:00.408359 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d91b1cae-b9ac-432e-ad1c-afae4f6d2c6b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:00.409887 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d91b1cae-b9ac-432e-ad1c-afae4f6d2c6b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:00.411041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d91b1cae-b9ac-432e-ad1c-afae4f6d2c6b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1725 microversion: 2.20 time: 0.148644 Jul 05 10:24:00.411595 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 466/958] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:00 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1725 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:01.428193 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c075f5fb-269e-44fe-a266-cc8e0dc7812c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:01.428831 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c075f5fb-269e-44fe-a266-cc8e0dc7812c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:01.435274 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c075f5fb-269e-44fe-a266-cc8e0dc7812c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:01.435724 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c075f5fb-269e-44fe-a266-cc8e0dc7812c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:01.436516 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c075f5fb-269e-44fe-a266-cc8e0dc7812c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:01.436954 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c075f5fb-269e-44fe-a266-cc8e0dc7812c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:01.561949 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c075f5fb-269e-44fe-a266-cc8e0dc7812c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:01.569663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c075f5fb-269e-44fe-a266-cc8e0dc7812c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:01.571446 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c075f5fb-269e-44fe-a266-cc8e0dc7812c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:01.573119 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c075f5fb-269e-44fe-a266-cc8e0dc7812c tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1725 microversion: 2.20 time: 0.148330 Jul 05 10:24:01.573670 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 493/959] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:01 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1725 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:02.591585 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-1f360901-1399-410a-833c-c117a89126b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:02.592113 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-1f360901-1399-410a-833c-c117a89126b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:02.598808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1f360901-1399-410a-833c-c117a89126b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:02.599255 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1f360901-1399-410a-833c-c117a89126b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:02.600015 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1f360901-1399-410a-833c-c117a89126b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:02.600471 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1f360901-1399-410a-833c-c117a89126b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:02.730032 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1f360901-1399-410a-833c-c117a89126b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:02.737907 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1f360901-1399-410a-833c-c117a89126b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:02.739429 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1f360901-1399-410a-833c-c117a89126b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:02.740612 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-1f360901-1399-410a-833c-c117a89126b6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1725 microversion: 2.20 time: 0.153507 Jul 05 10:24:02.741160 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 467/960] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:02 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1725 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:03.758276 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-91d5afee-616b-4dee-81a1-5a97e4f37565 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:03.758870 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-91d5afee-616b-4dee-81a1-5a97e4f37565 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:03.765489 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-91d5afee-616b-4dee-81a1-5a97e4f37565 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:03.765931 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-91d5afee-616b-4dee-81a1-5a97e4f37565 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:03.766694 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-91d5afee-616b-4dee-81a1-5a97e4f37565 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:03.767129 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-91d5afee-616b-4dee-81a1-5a97e4f37565 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:03.895044 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-91d5afee-616b-4dee-81a1-5a97e4f37565 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:03.903320 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-91d5afee-616b-4dee-81a1-5a97e4f37565 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:03.904848 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-91d5afee-616b-4dee-81a1-5a97e4f37565 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:03.906003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-91d5afee-616b-4dee-81a1-5a97e4f37565 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1725 microversion: 2.20 time: 0.151626 Jul 05 10:24:03.906558 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 494/961] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:03 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:04.924714 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-30c8ed0e-57c3-4cb4-a038-2a4ca16e197f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:04.925323 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-30c8ed0e-57c3-4cb4-a038-2a4ca16e197f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:04.932334 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-30c8ed0e-57c3-4cb4-a038-2a4ca16e197f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:04.932809 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-30c8ed0e-57c3-4cb4-a038-2a4ca16e197f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:04.933571 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-30c8ed0e-57c3-4cb4-a038-2a4ca16e197f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:04.934003 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-30c8ed0e-57c3-4cb4-a038-2a4ca16e197f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:05.061446 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-30c8ed0e-57c3-4cb4-a038-2a4ca16e197f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:05.069232 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-30c8ed0e-57c3-4cb4-a038-2a4ca16e197f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:05.070998 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-30c8ed0e-57c3-4cb4-a038-2a4ca16e197f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:05.072312 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-30c8ed0e-57c3-4cb4-a038-2a4ca16e197f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1725 microversion: 2.20 time: 0.151635 Jul 05 10:24:05.072895 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 468/962] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:04 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:06.090837 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c8a7abd5-baa3-4c78-aa8f-cdb9e82c66e7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:06.091442 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c8a7abd5-baa3-4c78-aa8f-cdb9e82c66e7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:06.097955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c8a7abd5-baa3-4c78-aa8f-cdb9e82c66e7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:06.098400 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c8a7abd5-baa3-4c78-aa8f-cdb9e82c66e7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:06.099160 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c8a7abd5-baa3-4c78-aa8f-cdb9e82c66e7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:06.099591 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c8a7abd5-baa3-4c78-aa8f-cdb9e82c66e7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:06.226078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c8a7abd5-baa3-4c78-aa8f-cdb9e82c66e7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:06.233975 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c8a7abd5-baa3-4c78-aa8f-cdb9e82c66e7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:06.235462 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c8a7abd5-baa3-4c78-aa8f-cdb9e82c66e7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:06.236645 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c8a7abd5-baa3-4c78-aa8f-cdb9e82c66e7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1746 microversion: 2.20 time: 0.150265 Jul 05 10:24:06.237210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 495/963] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:06 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1746 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:07.255004 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d7e535ed-fea0-40cc-9602-22d4409891b9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:07.255606 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d7e535ed-fea0-40cc-9602-22d4409891b9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:07.262568 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d7e535ed-fea0-40cc-9602-22d4409891b9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:07.263021 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d7e535ed-fea0-40cc-9602-22d4409891b9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:07.263774 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d7e535ed-fea0-40cc-9602-22d4409891b9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:07.264209 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d7e535ed-fea0-40cc-9602-22d4409891b9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:07.389900 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d7e535ed-fea0-40cc-9602-22d4409891b9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:07.397670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d7e535ed-fea0-40cc-9602-22d4409891b9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:07.399276 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d7e535ed-fea0-40cc-9602-22d4409891b9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:07.401001 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d7e535ed-fea0-40cc-9602-22d4409891b9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1746 microversion: 2.20 time: 0.149913 Jul 05 10:24:07.401558 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 469/964] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:07 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1746 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:08.419449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-fdead1c6-ac67-4ef9-acdb-fb6778b3388a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:08.420053 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-fdead1c6-ac67-4ef9-acdb-fb6778b3388a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:08.426479 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fdead1c6-ac67-4ef9-acdb-fb6778b3388a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:08.426924 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fdead1c6-ac67-4ef9-acdb-fb6778b3388a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:08.427683 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fdead1c6-ac67-4ef9-acdb-fb6778b3388a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:08.428119 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fdead1c6-ac67-4ef9-acdb-fb6778b3388a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:08.450121 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-35422433-26bd-447b-9495-575397d1d9f6 req-3d0a60d6-442c-45ea-b5c8-ebf2be8ae480 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c90da36-1371-4861-9756-e275bf1f514a", "name": "network-vif-plugged", "status": "completed", "tag": "ceece817-3e21-46da-8f70-bbfe947c7cd6"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:24:08.459056 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-35422433-26bd-447b-9495-575397d1d9f6 req-3d0a60d6-442c-45ea-b5c8-ebf2be8ae480 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:08.459601 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-35422433-26bd-447b-9495-575397d1d9f6 req-3d0a60d6-442c-45ea-b5c8-ebf2be8ae480 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:08.471532 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-35422433-26bd-447b-9495-575397d1d9f6 req-3d0a60d6-442c-45ea-b5c8-ebf2be8ae480 service nova] Creating event network-vif-plugged:ceece817-3e21-46da-8f70-bbfe947c7cd6 for instance 1c90da36-1371-4861-9756-e275bf1f514a on hv-cinder-maste Jul 05 10:24:08.476650 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-35422433-26bd-447b-9495-575397d1d9f6 req-3d0a60d6-442c-45ea-b5c8-ebf2be8ae480 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030069 Jul 05 10:24:08.478330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 470/965] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:24:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:24:08.570769 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-fdead1c6-ac67-4ef9-acdb-fb6778b3388a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:08.579507 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-fdead1c6-ac67-4ef9-acdb-fb6778b3388a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:08.581056 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-fdead1c6-ac67-4ef9-acdb-fb6778b3388a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:08.582206 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-fdead1c6-ac67-4ef9-acdb-fb6778b3388a tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1746 microversion: 2.20 time: 0.167194 Jul 05 10:24:08.582774 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 496/966] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:08 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1746 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:09.602585 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-dda2e268-bd9e-4924-9c03-79ce225d030e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:09.603274 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-dda2e268-bd9e-4924-9c03-79ce225d030e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:09.609439 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dda2e268-bd9e-4924-9c03-79ce225d030e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:09.609886 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dda2e268-bd9e-4924-9c03-79ce225d030e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:09.610639 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dda2e268-bd9e-4924-9c03-79ce225d030e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:09.611073 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dda2e268-bd9e-4924-9c03-79ce225d030e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:09.740091 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-dda2e268-bd9e-4924-9c03-79ce225d030e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:09.748692 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-dda2e268-bd9e-4924-9c03-79ce225d030e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:09.750182 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-dda2e268-bd9e-4924-9c03-79ce225d030e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:09.751319 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-dda2e268-bd9e-4924-9c03-79ce225d030e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1746 microversion: 2.20 time: 0.153949 Jul 05 10:24:09.751873 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 471/967] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:09 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1746 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:10.768585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-838c4400-c193-4a66-bdb9-80f6e0419bf5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:10.769188 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-838c4400-c193-4a66-bdb9-80f6e0419bf5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:10.775577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-838c4400-c193-4a66-bdb9-80f6e0419bf5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:10.776024 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-838c4400-c193-4a66-bdb9-80f6e0419bf5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:10.776808 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-838c4400-c193-4a66-bdb9-80f6e0419bf5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:10.777245 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-838c4400-c193-4a66-bdb9-80f6e0419bf5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:11.144570 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-838c4400-c193-4a66-bdb9-80f6e0419bf5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:11.152369 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-838c4400-c193-4a66-bdb9-80f6e0419bf5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:11.153892 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-838c4400-c193-4a66-bdb9-80f6e0419bf5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:11.155032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-838c4400-c193-4a66-bdb9-80f6e0419bf5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1746 microversion: 2.20 time: 0.390346 Jul 05 10:24:11.155588 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 497/968] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:10 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1746 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:12.174069 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-87e7ea37-315a-440d-9164-15c106129aa3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:12.174696 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-87e7ea37-315a-440d-9164-15c106129aa3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:12.181601 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-87e7ea37-315a-440d-9164-15c106129aa3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:12.182045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-87e7ea37-315a-440d-9164-15c106129aa3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:12.182797 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-87e7ea37-315a-440d-9164-15c106129aa3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:12.183235 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-87e7ea37-315a-440d-9164-15c106129aa3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:12.317794 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-87e7ea37-315a-440d-9164-15c106129aa3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:12.327746 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-87e7ea37-315a-440d-9164-15c106129aa3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:12.329773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-87e7ea37-315a-440d-9164-15c106129aa3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:12.331264 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-87e7ea37-315a-440d-9164-15c106129aa3 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1741 microversion: 2.20 time: 0.161083 Jul 05 10:24:12.342583 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 472/969] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:12 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1741 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:13.349647 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-edaea065-3d3b-418a-882b-b7081a3b90ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:13.350226 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-edaea065-3d3b-418a-882b-b7081a3b90ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:13.357507 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-edaea065-3d3b-418a-882b-b7081a3b90ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:13.357975 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-edaea065-3d3b-418a-882b-b7081a3b90ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:13.358738 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-edaea065-3d3b-418a-882b-b7081a3b90ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:13.359168 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-edaea065-3d3b-418a-882b-b7081a3b90ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:13.500247 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-edaea065-3d3b-418a-882b-b7081a3b90ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:13.508790 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-edaea065-3d3b-418a-882b-b7081a3b90ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:13.510298 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-edaea065-3d3b-418a-882b-b7081a3b90ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:13.511492 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-edaea065-3d3b-418a-882b-b7081a3b90ce tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1741 microversion: 2.20 time: 0.165678 Jul 05 10:24:13.512239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 498/970] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:13 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1741 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:14.529582 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-76d91738-6e8b-47f9-9a25-7470e84e5dfc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:14.530175 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-76d91738-6e8b-47f9-9a25-7470e84e5dfc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:14.537693 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-76d91738-6e8b-47f9-9a25-7470e84e5dfc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:14.538976 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-76d91738-6e8b-47f9-9a25-7470e84e5dfc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:14.540027 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-76d91738-6e8b-47f9-9a25-7470e84e5dfc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:14.540682 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-76d91738-6e8b-47f9-9a25-7470e84e5dfc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:14.682395 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-76d91738-6e8b-47f9-9a25-7470e84e5dfc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:14.692530 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-76d91738-6e8b-47f9-9a25-7470e84e5dfc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:14.692530 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-76d91738-6e8b-47f9-9a25-7470e84e5dfc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:14.695413 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-76d91738-6e8b-47f9-9a25-7470e84e5dfc tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1741 microversion: 2.20 time: 0.168462 Jul 05 10:24:14.705272 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 473/971] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:14 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1741 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:15.712740 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f3c1d545-6089-4052-a66d-920480441c05 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:15.714168 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f3c1d545-6089-4052-a66d-920480441c05 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:15.721093 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f3c1d545-6089-4052-a66d-920480441c05 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:15.721888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f3c1d545-6089-4052-a66d-920480441c05 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:15.722641 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f3c1d545-6089-4052-a66d-920480441c05 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:15.723068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f3c1d545-6089-4052-a66d-920480441c05 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:15.882815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f3c1d545-6089-4052-a66d-920480441c05 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:15.891337 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f3c1d545-6089-4052-a66d-920480441c05 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:15.893011 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f3c1d545-6089-4052-a66d-920480441c05 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:15.894249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f3c1d545-6089-4052-a66d-920480441c05 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1741 microversion: 2.20 time: 0.185794 Jul 05 10:24:15.908216 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 499/972] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:15 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1741 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:16.916498 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-628d8483-02dd-4c5c-b1e2-9666e087be00 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:16.917469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-628d8483-02dd-4c5c-b1e2-9666e087be00 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:16.929081 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-628d8483-02dd-4c5c-b1e2-9666e087be00 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:16.930132 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-628d8483-02dd-4c5c-b1e2-9666e087be00 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:16.932803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-628d8483-02dd-4c5c-b1e2-9666e087be00 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:16.933752 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-628d8483-02dd-4c5c-b1e2-9666e087be00 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:17.098709 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-628d8483-02dd-4c5c-b1e2-9666e087be00 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:17.110004 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-628d8483-02dd-4c5c-b1e2-9666e087be00 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:17.111778 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-628d8483-02dd-4c5c-b1e2-9666e087be00 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:17.113693 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-628d8483-02dd-4c5c-b1e2-9666e087be00 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1741 microversion: 2.20 time: 0.202080 Jul 05 10:24:17.114453 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 474/973] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:16 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1741 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:18.135284 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1bc1e349-7409-488b-85d0-309cd568de6f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:18.136689 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1bc1e349-7409-488b-85d0-309cd568de6f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:18.146716 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1bc1e349-7409-488b-85d0-309cd568de6f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:18.147544 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1bc1e349-7409-488b-85d0-309cd568de6f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:18.148631 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1bc1e349-7409-488b-85d0-309cd568de6f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:18.149135 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1bc1e349-7409-488b-85d0-309cd568de6f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:18.299162 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1bc1e349-7409-488b-85d0-309cd568de6f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:18.309662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1bc1e349-7409-488b-85d0-309cd568de6f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:18.311276 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1bc1e349-7409-488b-85d0-309cd568de6f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:18.312463 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1bc1e349-7409-488b-85d0-309cd568de6f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1741 microversion: 2.20 time: 0.182000 Jul 05 10:24:18.316423 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 500/974] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:18 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1741 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:19.331852 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b128a9aa-5eb7-4fbe-9084-a4b9b517ec3f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:19.333819 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b128a9aa-5eb7-4fbe-9084-a4b9b517ec3f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:19.340897 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b128a9aa-5eb7-4fbe-9084-a4b9b517ec3f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:19.341371 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b128a9aa-5eb7-4fbe-9084-a4b9b517ec3f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:19.342360 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b128a9aa-5eb7-4fbe-9084-a4b9b517ec3f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:19.343276 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b128a9aa-5eb7-4fbe-9084-a4b9b517ec3f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:19.502742 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b128a9aa-5eb7-4fbe-9084-a4b9b517ec3f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:19.512304 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b128a9aa-5eb7-4fbe-9084-a4b9b517ec3f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:19.514454 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b128a9aa-5eb7-4fbe-9084-a4b9b517ec3f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:19.516143 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b128a9aa-5eb7-4fbe-9084-a4b9b517ec3f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1741 microversion: 2.20 time: 0.188846 Jul 05 10:24:19.528931 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 475/975] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:19 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1741 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:20.554964 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-17c27c19-27da-4858-a301-2920481f1770 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:20.555546 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-17c27c19-27da-4858-a301-2920481f1770 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:20.562231 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-17c27c19-27da-4858-a301-2920481f1770 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:20.562682 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-17c27c19-27da-4858-a301-2920481f1770 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:20.563447 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-17c27c19-27da-4858-a301-2920481f1770 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:20.563882 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-17c27c19-27da-4858-a301-2920481f1770 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:20.705268 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-17c27c19-27da-4858-a301-2920481f1770 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:20.714427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-17c27c19-27da-4858-a301-2920481f1770 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:20.716082 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-17c27c19-27da-4858-a301-2920481f1770 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:20.717388 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-17c27c19-27da-4858-a301-2920481f1770 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1741 microversion: 2.20 time: 0.170985 Jul 05 10:24:20.730521 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 501/976] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:20 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1741 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:21.737646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-1d8f5027-8359-4892-a04a-2440198dcb18 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:21.738258 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-1d8f5027-8359-4892-a04a-2440198dcb18 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:21.745351 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d8f5027-8359-4892-a04a-2440198dcb18 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:21.745800 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d8f5027-8359-4892-a04a-2440198dcb18 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:21.746560 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d8f5027-8359-4892-a04a-2440198dcb18 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:21.746998 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d8f5027-8359-4892-a04a-2440198dcb18 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:21.876684 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1d8f5027-8359-4892-a04a-2440198dcb18 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:21.884752 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1d8f5027-8359-4892-a04a-2440198dcb18 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:21.886251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1d8f5027-8359-4892-a04a-2440198dcb18 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:21.887435 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-1d8f5027-8359-4892-a04a-2440198dcb18 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1741 microversion: 2.20 time: 0.153654 Jul 05 10:24:21.887992 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 476/977] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:21 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1741 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:22.905691 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b547719e-617a-4fa9-a3a0-acf851b3b7b7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:22.906878 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b547719e-617a-4fa9-a3a0-acf851b3b7b7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:22.909678 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=27 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:24:22.915068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b547719e-617a-4fa9-a3a0-acf851b3b7b7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:22.915513 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b547719e-617a-4fa9-a3a0-acf851b3b7b7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:22.916276 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b547719e-617a-4fa9-a3a0-acf851b3b7b7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:22.916750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b547719e-617a-4fa9-a3a0-acf851b3b7b7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:22.921022 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=77 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:24:23.063172 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b547719e-617a-4fa9-a3a0-acf851b3b7b7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:23.071325 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b547719e-617a-4fa9-a3a0-acf851b3b7b7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:23.072848 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b547719e-617a-4fa9-a3a0-acf851b3b7b7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:23.074007 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b547719e-617a-4fa9-a3a0-acf851b3b7b7 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1723 microversion: 2.20 time: 0.172092 Jul 05 10:24:23.075468 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 502/978] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:22 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1723 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:24.094299 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-904febca-dff4-4a6b-a6ae-7bc17c736480 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:24.094911 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-904febca-dff4-4a6b-a6ae-7bc17c736480 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:24.101491 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-904febca-dff4-4a6b-a6ae-7bc17c736480 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:24.101949 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-904febca-dff4-4a6b-a6ae-7bc17c736480 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:24.102708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-904febca-dff4-4a6b-a6ae-7bc17c736480 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:24.103146 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-904febca-dff4-4a6b-a6ae-7bc17c736480 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:24.231087 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-904febca-dff4-4a6b-a6ae-7bc17c736480 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:24.233385 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:24:24.241066 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-904febca-dff4-4a6b-a6ae-7bc17c736480 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:24.242561 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-904febca-dff4-4a6b-a6ae-7bc17c736480 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:24.243713 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-904febca-dff4-4a6b-a6ae-7bc17c736480 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1723 microversion: 2.20 time: 0.153341 Jul 05 10:24:24.244267 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 477/979] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:24 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:24.525369 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-19b119a7-add4-4b02-be47-a11331e062ad req-ccac0c4e-52ba-4a80-bee8-61c950212897 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c90da36-1371-4861-9756-e275bf1f514a", "name": "network-vif-unplugged", "status": "completed", "tag": "ceece817-3e21-46da-8f70-bbfe947c7cd6"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:24:24.533887 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-19b119a7-add4-4b02-be47-a11331e062ad req-ccac0c4e-52ba-4a80-bee8-61c950212897 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:24.534327 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-19b119a7-add4-4b02-be47-a11331e062ad req-ccac0c4e-52ba-4a80-bee8-61c950212897 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:24.546366 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-19b119a7-add4-4b02-be47-a11331e062ad req-ccac0c4e-52ba-4a80-bee8-61c950212897 service nova] Creating event network-vif-unplugged:ceece817-3e21-46da-8f70-bbfe947c7cd6 for instance 1c90da36-1371-4861-9756-e275bf1f514a on hv-cinder-maste Jul 05 10:24:24.551322 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-19b119a7-add4-4b02-be47-a11331e062ad req-ccac0c4e-52ba-4a80-bee8-61c950212897 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.029542 Jul 05 10:24:24.552526 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 503/980] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:24:24 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:24:25.261606 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3c015676-0527-4bae-b12c-db7473c57e9d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:25.262217 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3c015676-0527-4bae-b12c-db7473c57e9d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:25.264848 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=31 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:24:25.273619 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3c015676-0527-4bae-b12c-db7473c57e9d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:25.274264 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3c015676-0527-4bae-b12c-db7473c57e9d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:25.275047 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3c015676-0527-4bae-b12c-db7473c57e9d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:25.275490 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3c015676-0527-4bae-b12c-db7473c57e9d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:25.450409 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-3c015676-0527-4bae-b12c-db7473c57e9d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:25.459670 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-3c015676-0527-4bae-b12c-db7473c57e9d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:25.461245 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-3c015676-0527-4bae-b12c-db7473c57e9d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:25.462406 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3c015676-0527-4bae-b12c-db7473c57e9d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1726 microversion: 2.20 time: 0.204717 Jul 05 10:24:25.463184 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 478/981] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:25 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1726 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:25.480593 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c9b293f6-2fa9-4a05-a6d1-0f14fb0c692f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b4748f8e-c790-4afa-a7b6-4b9f9a28fa3e"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:24:25.481821 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c9b293f6-2fa9-4a05-a6d1-0f14fb0c692f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:25.488078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c9b293f6-2fa9-4a05-a6d1-0f14fb0c692f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:25.488622 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c9b293f6-2fa9-4a05-a6d1-0f14fb0c692f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:25.489443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c9b293f6-2fa9-4a05-a6d1-0f14fb0c692f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:25.489877 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c9b293f6-2fa9-4a05-a6d1-0f14fb0c692f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:25.753296 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c9b293f6-2fa9-4a05-a6d1-0f14fb0c692f tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "POST /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.277136 Jul 05 10:24:25.754379 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 504/982] 10.222.0.20 () {62 vars in 1363 bytes} [Tue Jul 5 10:24:25 2022] POST /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/os-volume_attachments => generated 188 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 05 10:24:25.869218 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3602133f-30e5-42de-ace6-be4b238dca2d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:25.869919 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3602133f-30e5-42de-ace6-be4b238dca2d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:25.875924 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3602133f-30e5-42de-ace6-be4b238dca2d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:25.876521 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3602133f-30e5-42de-ace6-be4b238dca2d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:25.877359 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3602133f-30e5-42de-ace6-be4b238dca2d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:25.877803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3602133f-30e5-42de-ace6-be4b238dca2d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:26.123604 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3602133f-30e5-42de-ace6-be4b238dca2d tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "DELETE /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/os-volume_attachments/b4748f8e-c790-4afa-a7b6-4b9f9a28fa3e" status: 202 len: 0 microversion: 2.20 time: 0.258160 Jul 05 10:24:26.124193 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 479/983] 10.222.0.20 () {60 vars in 1456 bytes} [Tue Jul 5 10:24:25 2022] DELETE /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/os-volume_attachments/b4748f8e-c790-4afa-a7b6-4b9f9a28fa3e => generated 0 bytes in 260 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Jul 05 10:24:26.178250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-52134572-5620-44ea-9188-991e3023a854 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:24:26.178803 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-52134572-5620-44ea-9188-991e3023a854 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:26.184777 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-52134572-5620-44ea-9188-991e3023a854 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:26.185217 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-52134572-5620-44ea-9188-991e3023a854 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:26.185961 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-52134572-5620-44ea-9188-991e3023a854 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:26.186397 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-52134572-5620-44ea-9188-991e3023a854 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:26.278750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-52134572-5620-44ea-9188-991e3023a854 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "POST /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/action" status: 202 len: 0 microversion: 2.20 time: 0.104852 Jul 05 10:24:26.279507 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 505/984] 10.222.0.20 () {62 vars in 1318 bytes} [Tue Jul 5 10:24:26 2022] POST /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/action => generated 0 bytes in 106 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:24:26.288041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5000d294-ce99-45ce-b2da-96556c3fe2fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:26.288780 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5000d294-ce99-45ce-b2da-96556c3fe2fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:26.295997 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5000d294-ce99-45ce-b2da-96556c3fe2fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:26.296699 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5000d294-ce99-45ce-b2da-96556c3fe2fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:26.297482 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5000d294-ce99-45ce-b2da-96556c3fe2fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:26.298174 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5000d294-ce99-45ce-b2da-96556c3fe2fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:26.430560 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5000d294-ce99-45ce-b2da-96556c3fe2fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:26.438356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5000d294-ce99-45ce-b2da-96556c3fe2fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:26.439859 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5000d294-ce99-45ce-b2da-96556c3fe2fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:26.441326 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5000d294-ce99-45ce-b2da-96556c3fe2fe tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1674 microversion: 2.20 time: 0.157563 Jul 05 10:24:26.441883 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 480/985] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:26 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1674 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:27.460612 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-69a0eb9c-d7a2-4be8-9f13-49deb43db5c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:27.461217 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-69a0eb9c-d7a2-4be8-9f13-49deb43db5c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:27.467615 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-69a0eb9c-d7a2-4be8-9f13-49deb43db5c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:27.468063 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-69a0eb9c-d7a2-4be8-9f13-49deb43db5c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:27.468865 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-69a0eb9c-d7a2-4be8-9f13-49deb43db5c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:27.469302 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-69a0eb9c-d7a2-4be8-9f13-49deb43db5c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:27.649449 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-69a0eb9c-d7a2-4be8-9f13-49deb43db5c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:27.657412 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-69a0eb9c-d7a2-4be8-9f13-49deb43db5c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:27.658900 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-69a0eb9c-d7a2-4be8-9f13-49deb43db5c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:27.660057 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-69a0eb9c-d7a2-4be8-9f13-49deb43db5c8 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.204086 Jul 05 10:24:27.661158 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 506/986] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:27 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:28.680528 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b973a14d-5e6e-4416-bf76-8a0d01e6c0ee tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:28.681758 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b973a14d-5e6e-4416-bf76-8a0d01e6c0ee tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:28.688802 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b973a14d-5e6e-4416-bf76-8a0d01e6c0ee tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:28.689254 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b973a14d-5e6e-4416-bf76-8a0d01e6c0ee tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:28.690026 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b973a14d-5e6e-4416-bf76-8a0d01e6c0ee tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:28.690938 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b973a14d-5e6e-4416-bf76-8a0d01e6c0ee tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:28.848407 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b973a14d-5e6e-4416-bf76-8a0d01e6c0ee tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:28.856850 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b973a14d-5e6e-4416-bf76-8a0d01e6c0ee tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:28.858615 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b973a14d-5e6e-4416-bf76-8a0d01e6c0ee tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:28.860089 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b973a14d-5e6e-4416-bf76-8a0d01e6c0ee tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.184495 Jul 05 10:24:28.872304 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 481/987] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:28 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:29.878511 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-4f1100d0-6500-4030-a24d-f10a35d0a449 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:29.879119 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-4f1100d0-6500-4030-a24d-f10a35d0a449 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:29.885731 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f1100d0-6500-4030-a24d-f10a35d0a449 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:29.886169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f1100d0-6500-4030-a24d-f10a35d0a449 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:29.886935 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f1100d0-6500-4030-a24d-f10a35d0a449 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:29.887363 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f1100d0-6500-4030-a24d-f10a35d0a449 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:30.027350 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4f1100d0-6500-4030-a24d-f10a35d0a449 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:30.036094 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4f1100d0-6500-4030-a24d-f10a35d0a449 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:30.037622 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4f1100d0-6500-4030-a24d-f10a35d0a449 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:30.038785 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-4f1100d0-6500-4030-a24d-f10a35d0a449 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.164138 Jul 05 10:24:30.039341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 507/988] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:29 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:31.056669 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-256a4652-6a0b-4a83-a1fb-fa8129811f61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:31.057516 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-256a4652-6a0b-4a83-a1fb-fa8129811f61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:31.063565 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-256a4652-6a0b-4a83-a1fb-fa8129811f61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:31.064009 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-256a4652-6a0b-4a83-a1fb-fa8129811f61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:31.064814 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-256a4652-6a0b-4a83-a1fb-fa8129811f61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:31.065518 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-256a4652-6a0b-4a83-a1fb-fa8129811f61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:31.202112 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-256a4652-6a0b-4a83-a1fb-fa8129811f61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:31.211246 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-256a4652-6a0b-4a83-a1fb-fa8129811f61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:31.212808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-256a4652-6a0b-4a83-a1fb-fa8129811f61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:31.213978 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-256a4652-6a0b-4a83-a1fb-fa8129811f61 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.161485 Jul 05 10:24:31.214575 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 482/989] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:31 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 05 10:24:32.233311 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0e2e78d5-4ae6-4014-b0db-cd74ec8b5b89 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:32.233922 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0e2e78d5-4ae6-4014-b0db-cd74ec8b5b89 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:32.241024 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0e2e78d5-4ae6-4014-b0db-cd74ec8b5b89 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:32.241473 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0e2e78d5-4ae6-4014-b0db-cd74ec8b5b89 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:32.242238 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0e2e78d5-4ae6-4014-b0db-cd74ec8b5b89 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:32.242672 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0e2e78d5-4ae6-4014-b0db-cd74ec8b5b89 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:32.381575 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0e2e78d5-4ae6-4014-b0db-cd74ec8b5b89 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:32.390019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0e2e78d5-4ae6-4014-b0db-cd74ec8b5b89 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:32.391496 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0e2e78d5-4ae6-4014-b0db-cd74ec8b5b89 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:32.392667 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0e2e78d5-4ae6-4014-b0db-cd74ec8b5b89 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.163233 Jul 05 10:24:32.393234 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 508/990] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:32 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:33.411884 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-90bb3813-8d2a-46ef-a456-63bb6829290e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:33.412447 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-90bb3813-8d2a-46ef-a456-63bb6829290e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:33.419041 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-90bb3813-8d2a-46ef-a456-63bb6829290e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:33.419495 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-90bb3813-8d2a-46ef-a456-63bb6829290e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:33.420272 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-90bb3813-8d2a-46ef-a456-63bb6829290e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:33.420743 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-90bb3813-8d2a-46ef-a456-63bb6829290e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:33.563579 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-90bb3813-8d2a-46ef-a456-63bb6829290e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:33.572268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-90bb3813-8d2a-46ef-a456-63bb6829290e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:33.573923 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-90bb3813-8d2a-46ef-a456-63bb6829290e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:33.575102 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-90bb3813-8d2a-46ef-a456-63bb6829290e tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.167446 Jul 05 10:24:33.575659 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 483/991] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:33 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 05 10:24:34.594144 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1cca52ba-fc07-4cfa-95ba-f72ad7ed9284 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:34.594837 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1cca52ba-fc07-4cfa-95ba-f72ad7ed9284 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:34.600833 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1cca52ba-fc07-4cfa-95ba-f72ad7ed9284 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:34.601280 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1cca52ba-fc07-4cfa-95ba-f72ad7ed9284 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:34.602049 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1cca52ba-fc07-4cfa-95ba-f72ad7ed9284 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:34.602486 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1cca52ba-fc07-4cfa-95ba-f72ad7ed9284 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:34.746096 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1cca52ba-fc07-4cfa-95ba-f72ad7ed9284 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:34.755127 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1cca52ba-fc07-4cfa-95ba-f72ad7ed9284 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:34.756708 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1cca52ba-fc07-4cfa-95ba-f72ad7ed9284 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:34.757903 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1cca52ba-fc07-4cfa-95ba-f72ad7ed9284 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.168299 Jul 05 10:24:34.758549 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 509/992] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:34 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:35.775098 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e406ab2a-cc34-4cc7-ac4f-9a4d5a222c1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:35.776510 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e406ab2a-cc34-4cc7-ac4f-9a4d5a222c1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:35.783528 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e406ab2a-cc34-4cc7-ac4f-9a4d5a222c1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:35.784072 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e406ab2a-cc34-4cc7-ac4f-9a4d5a222c1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:35.784946 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e406ab2a-cc34-4cc7-ac4f-9a4d5a222c1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:35.785460 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e406ab2a-cc34-4cc7-ac4f-9a4d5a222c1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:35.920308 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e406ab2a-cc34-4cc7-ac4f-9a4d5a222c1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:35.929800 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e406ab2a-cc34-4cc7-ac4f-9a4d5a222c1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:35.930752 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e406ab2a-cc34-4cc7-ac4f-9a4d5a222c1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:35.931923 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e406ab2a-cc34-4cc7-ac4f-9a4d5a222c1b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.160768 Jul 05 10:24:35.932855 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 484/993] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:35 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:36.951027 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-bf9b17d6-9b37-4e0c-be6a-77ccd3f0ed48 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:36.951635 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-bf9b17d6-9b37-4e0c-be6a-77ccd3f0ed48 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:36.959646 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bf9b17d6-9b37-4e0c-be6a-77ccd3f0ed48 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:36.960150 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bf9b17d6-9b37-4e0c-be6a-77ccd3f0ed48 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:36.961192 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bf9b17d6-9b37-4e0c-be6a-77ccd3f0ed48 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:36.961825 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-bf9b17d6-9b37-4e0c-be6a-77ccd3f0ed48 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:37.092637 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-bf9b17d6-9b37-4e0c-be6a-77ccd3f0ed48 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:37.101001 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-bf9b17d6-9b37-4e0c-be6a-77ccd3f0ed48 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:37.102504 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-bf9b17d6-9b37-4e0c-be6a-77ccd3f0ed48 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:37.103658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-bf9b17d6-9b37-4e0c-be6a-77ccd3f0ed48 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.156544 Jul 05 10:24:37.104225 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 510/994] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:36 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:38.121064 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b762e3d3-403b-4894-a6f9-bed4d7987211 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:38.121684 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b762e3d3-403b-4894-a6f9-bed4d7987211 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:38.128653 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b762e3d3-403b-4894-a6f9-bed4d7987211 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:38.129104 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b762e3d3-403b-4894-a6f9-bed4d7987211 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:38.129863 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b762e3d3-403b-4894-a6f9-bed4d7987211 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:38.130298 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b762e3d3-403b-4894-a6f9-bed4d7987211 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:38.257570 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b762e3d3-403b-4894-a6f9-bed4d7987211 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:38.265330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b762e3d3-403b-4894-a6f9-bed4d7987211 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:38.266819 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b762e3d3-403b-4894-a6f9-bed4d7987211 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:38.268780 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b762e3d3-403b-4894-a6f9-bed4d7987211 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.151425 Jul 05 10:24:38.269160 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 485/995] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:38 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:39.287477 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d3d715ea-3b3a-4edf-bc0a-65e663fd599b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:39.288110 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d3d715ea-3b3a-4edf-bc0a-65e663fd599b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:39.294593 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d3d715ea-3b3a-4edf-bc0a-65e663fd599b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:39.295040 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d3d715ea-3b3a-4edf-bc0a-65e663fd599b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:39.295796 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d3d715ea-3b3a-4edf-bc0a-65e663fd599b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:39.296232 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d3d715ea-3b3a-4edf-bc0a-65e663fd599b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:39.430261 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d3d715ea-3b3a-4edf-bc0a-65e663fd599b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:39.438863 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d3d715ea-3b3a-4edf-bc0a-65e663fd599b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:39.440372 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d3d715ea-3b3a-4edf-bc0a-65e663fd599b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:39.441568 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d3d715ea-3b3a-4edf-bc0a-65e663fd599b tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.158617 Jul 05 10:24:39.442160 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 511/996] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:39 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:40.459185 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f8630c9f-431a-482f-a42f-853d09efacef tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:40.459796 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f8630c9f-431a-482f-a42f-853d09efacef tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:40.466393 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f8630c9f-431a-482f-a42f-853d09efacef tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:40.466841 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f8630c9f-431a-482f-a42f-853d09efacef tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:40.467600 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f8630c9f-431a-482f-a42f-853d09efacef tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:40.468039 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f8630c9f-431a-482f-a42f-853d09efacef tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:40.598131 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f8630c9f-431a-482f-a42f-853d09efacef tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:40.605891 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f8630c9f-431a-482f-a42f-853d09efacef tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:40.607408 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f8630c9f-431a-482f-a42f-853d09efacef tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:40.608574 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f8630c9f-431a-482f-a42f-853d09efacef tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.153326 Jul 05 10:24:40.609134 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 486/997] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:40 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:41.627771 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0dcf5eb4-c09a-43c7-a701-06f0c4d1f7e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:41.628416 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0dcf5eb4-c09a-43c7-a701-06f0c4d1f7e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:41.634845 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0dcf5eb4-c09a-43c7-a701-06f0c4d1f7e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:41.635283 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0dcf5eb4-c09a-43c7-a701-06f0c4d1f7e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:41.636028 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0dcf5eb4-c09a-43c7-a701-06f0c4d1f7e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:41.636485 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-0dcf5eb4-c09a-43c7-a701-06f0c4d1f7e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:41.761395 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0dcf5eb4-c09a-43c7-a701-06f0c4d1f7e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:41.769272 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0dcf5eb4-c09a-43c7-a701-06f0c4d1f7e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:41.770685 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0dcf5eb4-c09a-43c7-a701-06f0c4d1f7e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:41.771822 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0dcf5eb4-c09a-43c7-a701-06f0c4d1f7e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.148619 Jul 05 10:24:41.772374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 512/998] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:41 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:42.789880 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-583c9223-82cc-4212-b6e5-87ce3c7c4342 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:42.790430 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-583c9223-82cc-4212-b6e5-87ce3c7c4342 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:42.796896 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-583c9223-82cc-4212-b6e5-87ce3c7c4342 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:42.797351 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-583c9223-82cc-4212-b6e5-87ce3c7c4342 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:42.798120 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-583c9223-82cc-4212-b6e5-87ce3c7c4342 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:42.798562 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-583c9223-82cc-4212-b6e5-87ce3c7c4342 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:42.928130 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-583c9223-82cc-4212-b6e5-87ce3c7c4342 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:42.936267 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-583c9223-82cc-4212-b6e5-87ce3c7c4342 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:42.937807 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-583c9223-82cc-4212-b6e5-87ce3c7c4342 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:42.939811 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-583c9223-82cc-4212-b6e5-87ce3c7c4342 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.153475 Jul 05 10:24:42.940382 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 487/999] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:42 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:43.958794 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c743a460-cd8e-4b0a-a020-42f33174e7f5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:43.959393 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c743a460-cd8e-4b0a-a020-42f33174e7f5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:43.965751 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c743a460-cd8e-4b0a-a020-42f33174e7f5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:43.966194 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c743a460-cd8e-4b0a-a020-42f33174e7f5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:43.966953 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c743a460-cd8e-4b0a-a020-42f33174e7f5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:43.967378 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c743a460-cd8e-4b0a-a020-42f33174e7f5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:44.094047 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c743a460-cd8e-4b0a-a020-42f33174e7f5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:44.101954 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c743a460-cd8e-4b0a-a020-42f33174e7f5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:44.103444 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c743a460-cd8e-4b0a-a020-42f33174e7f5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:44.104635 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c743a460-cd8e-4b0a-a020-42f33174e7f5 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.149662 Jul 05 10:24:44.105199 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 513/1000] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:43 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:44.687269 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-97f5c6bd-d41e-4b88-ac66-cdc49b66c31a req-3b0aa5ff-7a6c-47be-b1ad-6e6ce338f766 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c90da36-1371-4861-9756-e275bf1f514a", "name": "network-vif-plugged", "status": "completed", "tag": "ceece817-3e21-46da-8f70-bbfe947c7cd6"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:24:44.696657 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-97f5c6bd-d41e-4b88-ac66-cdc49b66c31a req-3b0aa5ff-7a6c-47be-b1ad-6e6ce338f766 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:44.697164 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-97f5c6bd-d41e-4b88-ac66-cdc49b66c31a req-3b0aa5ff-7a6c-47be-b1ad-6e6ce338f766 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:44.708634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-97f5c6bd-d41e-4b88-ac66-cdc49b66c31a req-3b0aa5ff-7a6c-47be-b1ad-6e6ce338f766 service nova] Creating event network-vif-plugged:ceece817-3e21-46da-8f70-bbfe947c7cd6 for instance 1c90da36-1371-4861-9756-e275bf1f514a on hv-cinder-maste Jul 05 10:24:44.713730 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-97f5c6bd-d41e-4b88-ac66-cdc49b66c31a req-3b0aa5ff-7a6c-47be-b1ad-6e6ce338f766 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030106 Jul 05 10:24:44.714412 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 488/1001] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:24:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:24:45.122558 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1b36e3d3-dc38-4175-a263-d307892b66ae tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:45.123140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1b36e3d3-dc38-4175-a263-d307892b66ae tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:45.129539 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1b36e3d3-dc38-4175-a263-d307892b66ae tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:45.129982 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1b36e3d3-dc38-4175-a263-d307892b66ae tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:45.130745 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1b36e3d3-dc38-4175-a263-d307892b66ae tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:45.131165 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1b36e3d3-dc38-4175-a263-d307892b66ae tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:45.259487 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1b36e3d3-dc38-4175-a263-d307892b66ae tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:45.267715 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1b36e3d3-dc38-4175-a263-d307892b66ae tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:45.269247 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1b36e3d3-dc38-4175-a263-d307892b66ae tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:45.270373 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1b36e3d3-dc38-4175-a263-d307892b66ae tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.151704 Jul 05 10:24:45.270922 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 514/1002] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:45 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:46.289419 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-1b37a953-0467-45d8-bcc0-2ad3e4650050 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:46.290023 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-1b37a953-0467-45d8-bcc0-2ad3e4650050 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:46.296239 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1b37a953-0467-45d8-bcc0-2ad3e4650050 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:46.296718 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1b37a953-0467-45d8-bcc0-2ad3e4650050 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:46.298081 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1b37a953-0467-45d8-bcc0-2ad3e4650050 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:46.298514 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1b37a953-0467-45d8-bcc0-2ad3e4650050 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:46.428131 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1b37a953-0467-45d8-bcc0-2ad3e4650050 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:46.435953 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1b37a953-0467-45d8-bcc0-2ad3e4650050 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:46.437753 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1b37a953-0467-45d8-bcc0-2ad3e4650050 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:46.438888 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-1b37a953-0467-45d8-bcc0-2ad3e4650050 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1732 microversion: 2.20 time: 0.154342 Jul 05 10:24:46.439451 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 489/1003] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:46 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1732 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:24:47.456853 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d8dcbfe9-b119-46ed-a8cb-4b1d55b97932 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:24:47.457465 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d8dcbfe9-b119-46ed-a8cb-4b1d55b97932 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:24:47.463968 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8dcbfe9-b119-46ed-a8cb-4b1d55b97932 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:47.464441 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8dcbfe9-b119-46ed-a8cb-4b1d55b97932 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:47.465206 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8dcbfe9-b119-46ed-a8cb-4b1d55b97932 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:24:47.465641 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d8dcbfe9-b119-46ed-a8cb-4b1d55b97932 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:24:47.594208 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d8dcbfe9-b119-46ed-a8cb-4b1d55b97932 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:47.596593 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:24:47.603547 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d8dcbfe9-b119-46ed-a8cb-4b1d55b97932 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:47.605072 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d8dcbfe9-b119-46ed-a8cb-4b1d55b97932 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:24:47.606221 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d8dcbfe9-b119-46ed-a8cb-4b1d55b97932 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1719 microversion: 2.20 time: 0.153219 Jul 05 10:24:47.606785 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 515/1004] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:24:47 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1719 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:25:20.580027 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:25:20.580649 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Tue Jul 5 10:25:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:25:27.038174 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:25:27.038773 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Tue Jul 5 10:25:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:25:35.230788 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-266c6cea-b8ae-4f41-b6fd-0a2b568b45f5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:35.231398 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-266c6cea-b8ae-4f41-b6fd-0a2b568b45f5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:35.233295 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=23 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:25:35.233935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=78,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:25:35.240125 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-266c6cea-b8ae-4f41-b6fd-0a2b568b45f5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:35.240609 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-266c6cea-b8ae-4f41-b6fd-0a2b568b45f5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:35.241369 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-266c6cea-b8ae-4f41-b6fd-0a2b568b45f5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:35.241803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-266c6cea-b8ae-4f41-b6fd-0a2b568b45f5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:35.267221 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-266c6cea-b8ae-4f41-b6fd-0a2b568b45f5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Going to try to terminate instance {{(pid=117812) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:25:35.342796 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-266c6cea-b8ae-4f41-b6fd-0a2b568b45f5 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "DELETE /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 204 len: 0 microversion: 2.1 time: 0.116368 Jul 05 10:25:35.343474 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 490/1005] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:25:35 2022] DELETE /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:25:35.352775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3083b379-37bf-4f50-acaf-80c475c2ea7a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:35.353908 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3083b379-37bf-4f50-acaf-80c475c2ea7a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:35.356487 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=29 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:25:35.363493 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3083b379-37bf-4f50-acaf-80c475c2ea7a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:35.363947 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3083b379-37bf-4f50-acaf-80c475c2ea7a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:35.364810 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3083b379-37bf-4f50-acaf-80c475c2ea7a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:35.365262 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3083b379-37bf-4f50-acaf-80c475c2ea7a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:35.561795 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-3083b379-37bf-4f50-acaf-80c475c2ea7a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:35.569376 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3083b379-37bf-4f50-acaf-80c475c2ea7a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1578 microversion: 2.1 time: 0.220907 Jul 05 10:25:35.569942 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 516/1006] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:35 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1578 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:36.173588 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-79ddb2a3-b236-42bb-88cd-3cc8478b6550 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:36.174593 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-79ddb2a3-b236-42bb-88cd-3cc8478b6550 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:36.181340 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-79ddb2a3-b236-42bb-88cd-3cc8478b6550 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:36.181735 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-79ddb2a3-b236-42bb-88cd-3cc8478b6550 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:36.182497 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-79ddb2a3-b236-42bb-88cd-3cc8478b6550 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:36.182936 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-79ddb2a3-b236-42bb-88cd-3cc8478b6550 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:36.211726 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-79ddb2a3-b236-42bb-88cd-3cc8478b6550 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.042401 Jul 05 10:25:36.212290 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 491/1007] 10.222.0.20 () {60 vars in 1342 bytes} [Tue Jul 5 10:25:36 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:36.268186 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-879f0ab0-2624-4aba-8da4-fbd345019913 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:36.268706 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-879f0ab0-2624-4aba-8da4-fbd345019913 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:36.274990 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-879f0ab0-2624-4aba-8da4-fbd345019913 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:36.275688 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-879f0ab0-2624-4aba-8da4-fbd345019913 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:36.276533 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-879f0ab0-2624-4aba-8da4-fbd345019913 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:36.276966 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-879f0ab0-2624-4aba-8da4-fbd345019913 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:36.300697 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-879f0ab0-2624-4aba-8da4-fbd345019913 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Going to try to terminate instance {{(pid=117813) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:25:36.372781 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-879f0ab0-2624-4aba-8da4-fbd345019913 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "DELETE /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 204 len: 0 microversion: 2.20 time: 0.108099 Jul 05 10:25:36.373491 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 517/1008] 10.222.0.20 () {60 vars in 1279 bytes} [Tue Jul 5 10:25:36 2022] DELETE /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 05 10:25:36.381475 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-1c1e520f-7b05-44a7-9fde-173460887a44 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:36.382087 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-1c1e520f-7b05-44a7-9fde-173460887a44 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:36.389310 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1c1e520f-7b05-44a7-9fde-173460887a44 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:36.389764 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1c1e520f-7b05-44a7-9fde-173460887a44 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:36.390521 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1c1e520f-7b05-44a7-9fde-173460887a44 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:36.390958 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1c1e520f-7b05-44a7-9fde-173460887a44 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:36.572845 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1c1e520f-7b05-44a7-9fde-173460887a44 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:36.580639 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1c1e520f-7b05-44a7-9fde-173460887a44 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:36.582166 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1c1e520f-7b05-44a7-9fde-173460887a44 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:36.583315 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-1c1e520f-7b05-44a7-9fde-173460887a44 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1725 microversion: 2.20 time: 0.205718 Jul 05 10:25:36.583865 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 492/1009] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:25:36 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1725 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:25:36.586210 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3c2e2eda-e4ba-4af3-9de3-55bd70964ad0 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:36.586957 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3c2e2eda-e4ba-4af3-9de3-55bd70964ad0 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:36.593440 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3c2e2eda-e4ba-4af3-9de3-55bd70964ad0 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:36.593886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3c2e2eda-e4ba-4af3-9de3-55bd70964ad0 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:36.594650 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3c2e2eda-e4ba-4af3-9de3-55bd70964ad0 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:36.595085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3c2e2eda-e4ba-4af3-9de3-55bd70964ad0 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:36.870648 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-3c2e2eda-e4ba-4af3-9de3-55bd70964ad0 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:36.878113 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3c2e2eda-e4ba-4af3-9de3-55bd70964ad0 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1578 microversion: 2.1 time: 0.296009 Jul 05 10:25:36.878668 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 518/1010] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:36 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1578 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:37.601946 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-1d261944-b889-425b-a60b-2b423b046917 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:37.602499 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-1d261944-b889-425b-a60b-2b423b046917 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:37.609058 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d261944-b889-425b-a60b-2b423b046917 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:37.609509 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d261944-b889-425b-a60b-2b423b046917 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:37.610268 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d261944-b889-425b-a60b-2b423b046917 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:37.610708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-1d261944-b889-425b-a60b-2b423b046917 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:37.738383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1d261944-b889-425b-a60b-2b423b046917 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:37.746777 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1d261944-b889-425b-a60b-2b423b046917 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:37.748288 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-1d261944-b889-425b-a60b-2b423b046917 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:37.749466 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-1d261944-b889-425b-a60b-2b423b046917 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1725 microversion: 2.20 time: 0.151432 Jul 05 10:25:37.750027 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 493/1011] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:25:37 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:25:37.895027 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8034a41e-3592-4050-a729-34263222487b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:37.895539 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8034a41e-3592-4050-a729-34263222487b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:37.902411 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8034a41e-3592-4050-a729-34263222487b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:37.902854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8034a41e-3592-4050-a729-34263222487b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:37.903606 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8034a41e-3592-4050-a729-34263222487b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:37.904035 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8034a41e-3592-4050-a729-34263222487b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:38.034850 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8034a41e-3592-4050-a729-34263222487b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:38.042410 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8034a41e-3592-4050-a729-34263222487b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1578 microversion: 2.1 time: 0.151322 Jul 05 10:25:38.042967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 519/1012] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:37 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1578 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:38.769088 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-af89404d-6c4d-48d9-b815-d694f8a15201 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:38.769699 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-af89404d-6c4d-48d9-b815-d694f8a15201 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:38.776837 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-af89404d-6c4d-48d9-b815-d694f8a15201 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:38.777298 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-af89404d-6c4d-48d9-b815-d694f8a15201 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:38.778060 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-af89404d-6c4d-48d9-b815-d694f8a15201 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:38.778505 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-af89404d-6c4d-48d9-b815-d694f8a15201 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:39.059735 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1dfee64c-c104-4457-8e19-1fff7a2fd840 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:39.060477 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1dfee64c-c104-4457-8e19-1fff7a2fd840 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:39.067155 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1dfee64c-c104-4457-8e19-1fff7a2fd840 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:39.067604 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1dfee64c-c104-4457-8e19-1fff7a2fd840 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:39.068380 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1dfee64c-c104-4457-8e19-1fff7a2fd840 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:39.068867 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1dfee64c-c104-4457-8e19-1fff7a2fd840 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:39.394746 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-af89404d-6c4d-48d9-b815-d694f8a15201 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:39.402608 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-af89404d-6c4d-48d9-b815-d694f8a15201 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:39.404110 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-af89404d-6c4d-48d9-b815-d694f8a15201 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:39.405297 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-af89404d-6c4d-48d9-b815-d694f8a15201 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1725 microversion: 2.20 time: 0.640172 Jul 05 10:25:39.405886 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 494/1013] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:25:38 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1725 bytes in 641 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:25:39.573357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1dfee64c-c104-4457-8e19-1fff7a2fd840 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:39.582133 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1dfee64c-c104-4457-8e19-1fff7a2fd840 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1578 microversion: 2.1 time: 0.526772 Jul 05 10:25:39.582914 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 520/1014] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:39 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1578 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:25:40.225600 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-f1f5ee58-7e16-439c-801d-7a6d670dcc46 req-1aa109fb-6479-4fdb-a755-057920ade5ac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c90da36-1371-4861-9756-e275bf1f514a", "name": "network-vif-deleted", "tag": "ceece817-3e21-46da-8f70-bbfe947c7cd6"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:25:40.244188 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-f1f5ee58-7e16-439c-801d-7a6d670dcc46 req-1aa109fb-6479-4fdb-a755-057920ade5ac service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:40.244448 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-f1f5ee58-7e16-439c-801d-7a6d670dcc46 req-1aa109fb-6479-4fdb-a755-057920ade5ac service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:40.263088 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-f1f5ee58-7e16-439c-801d-7a6d670dcc46 req-1aa109fb-6479-4fdb-a755-057920ade5ac service nova] Creating event network-vif-deleted:ceece817-3e21-46da-8f70-bbfe947c7cd6 for instance 1c90da36-1371-4861-9756-e275bf1f514a on hv-cinder-maste Jul 05 10:25:40.268808 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-f1f5ee58-7e16-439c-801d-7a6d670dcc46 req-1aa109fb-6479-4fdb-a755-057920ade5ac service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048285 Jul 05 10:25:40.270101 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 495/1015] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:25:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:25:40.425535 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-25c6a299-0d36-4f16-8365-1d67d15d8999 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:40.426583 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-25c6a299-0d36-4f16-8365-1d67d15d8999 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:40.434510 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-25c6a299-0d36-4f16-8365-1d67d15d8999 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:40.435319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-25c6a299-0d36-4f16-8365-1d67d15d8999 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:40.436510 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-25c6a299-0d36-4f16-8365-1d67d15d8999 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:40.437084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-25c6a299-0d36-4f16-8365-1d67d15d8999 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:40.478834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-25c6a299-0d36-4f16-8365-1d67d15d8999 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:40.479375 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-25c6a299-0d36-4f16-8365-1d67d15d8999 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:40.479902 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.objects.instance [None req-25c6a299-0d36-4f16-8365-1d67d15d8999 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lazy-loading 'fault' on Instance uuid 1c90da36-1371-4861-9756-e275bf1f514a {{(pid=117813) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:25:40.529698 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-25c6a299-0d36-4f16-8365-1d67d15d8999 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:40.538022 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-25c6a299-0d36-4f16-8365-1d67d15d8999 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:40.539571 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-25c6a299-0d36-4f16-8365-1d67d15d8999 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd4a608563189457dbcadf8d48acdd84a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f0c19d009f1b4ef09a40267bcf654107', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:40.540786 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-25c6a299-0d36-4f16-8365-1d67d15d8999 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 200 len: 1372 microversion: 2.20 time: 0.121526 Jul 05 10:25:40.541344 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 521/1016] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:25:40 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 1372 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 05 10:25:40.603109 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-43462ead-edb9-4f81-804a-708b58f17fd4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:40.603806 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-43462ead-edb9-4f81-804a-708b58f17fd4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:40.610007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-43462ead-edb9-4f81-804a-708b58f17fd4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:40.610458 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-43462ead-edb9-4f81-804a-708b58f17fd4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:40.611225 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-43462ead-edb9-4f81-804a-708b58f17fd4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:40.611664 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-43462ead-edb9-4f81-804a-708b58f17fd4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:40.691216 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-43462ead-edb9-4f81-804a-708b58f17fd4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:40.699217 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-43462ead-edb9-4f81-804a-708b58f17fd4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 200 len: 1226 microversion: 2.1 time: 0.100405 Jul 05 10:25:40.708342 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 496/1017] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:40 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 1226 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:41.558896 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-16974bdf-0053-47cc-938c-1fc8fca07de9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:41.559506 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-16974bdf-0053-47cc-938c-1fc8fca07de9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:41.565961 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-16974bdf-0053-47cc-938c-1fc8fca07de9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:41.566409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-16974bdf-0053-47cc-938c-1fc8fca07de9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:41.567169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-16974bdf-0053-47cc-938c-1fc8fca07de9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:41.567604 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-16974bdf-0053-47cc-938c-1fc8fca07de9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:41.580854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-16974bdf-0053-47cc-938c-1fc8fca07de9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] HTTP exception thrown: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-16974bdf-0053-47cc-938c-1fc8fca07de9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Request method failure captured: Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a HTTP/1.1 Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA_xvpEELgE-eBUYHUhNsBeyr4iL6TX2JC5RBVSqxhiIRu0uvV5DpIroGwyHi1VHjPe6ULauFPG_08J8dk6bV7uqHLXSCeuk3phF15RwcRNPncEKU39B23DXNZT1IGuIE1R6RvECOKMp30Qx65b7eEKmzJ9mKE-_AEJ6raY_zH75ufY Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Openstack-Nova-Api-Version: 2.20 Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: f0c19d009f1b4ef09a40267bcf654107 Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:25:41.582484 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:25:41.584250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f0c19d009f1b4ef09a40267bcf654107"}]}] Jul 05 10:25:41.584250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:41.584250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: f0c19d009f1b4ef09a40267bcf654107 Jul 05 10:25:41.584250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:41.584250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeShelveTestJSON-1813703734-project Jul 05 10:25:41.584250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:25:41.584250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:25:41.584250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: d4a608563189457dbcadf8d48acdd84a Jul 05 10:25:41.584250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1813703734-project Jul 05 10:25:41.584250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:25:41.584250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:41.584250 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': '1c90da36-1371-4861-9756-e275bf1f514a'} Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:25:41.585145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:41.587111 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:41.588277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-16974bdf-0053-47cc-938c-1fc8fca07de9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Returning 404 to user: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:25:41.588277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-16974bdf-0053-47cc-938c-1fc8fca07de9 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 404 len: 111 microversion: 2.20 time: 0.028914 Jul 05 10:25:41.588277 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 522/1018] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:25:41 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 05 10:25:41.593075 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7b302bc1-59ce-47fc-af53-6c3b620414c6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '.version_select at 0x7feb89783790>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:41.612972 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7b302bc1-59ce-47fc-af53-6c3b620414c6 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1899722997" status: 204 len: 0 microversion: 2.20 time: 0.023558 Jul 05 10:25:41.613461 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 497/1019] 10.222.0.20 () {60 vars in 1261 bytes} [Tue Jul 5 10:25:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1899722997 => generated 0 bytes in 25 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 05 10:25:41.715765 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6657933e-06d2-491e-8d06-04aa1ebc0305 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:41.716419 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-6657933e-06d2-491e-8d06-04aa1ebc0305 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:41.722378 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6657933e-06d2-491e-8d06-04aa1ebc0305 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:41.722823 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6657933e-06d2-491e-8d06-04aa1ebc0305 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:41.723635 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6657933e-06d2-491e-8d06-04aa1ebc0305 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:41.724182 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6657933e-06d2-491e-8d06-04aa1ebc0305 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:41.740295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-6657933e-06d2-491e-8d06-04aa1ebc0305 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] HTTP exception thrown: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6657933e-06d2-491e-8d06-04aa1ebc0305 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Request method failure captured: Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f HTTP/1.1 Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixBDpuHveZ3XeN9U67EK4lKdYSp_m_64mU-LBfdZJwGXE8VS4y9KkhlpV-sF73TZRPltmWaOCmj1JJgwzGBK1g-a6vXND_jKDWiyWKsXT9eD5CtGrsU53oavQhY5y4SU9PBsCzPb3si94ZYUpzEQAVskEwEVkAMCsD3v3D2ZU1cRNTYA Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 86eac45249ad4a5e9513fcc091bcf42e Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-TestVolumeBootPattern-620052980 Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:25:41.742123 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:25:41.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/86eac45249ad4a5e9513fcc091bcf42e"}]}] Jul 05 10:25:41.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-TestVolumeBootPattern-620052980 Jul 05 10:25:41.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 86eac45249ad4a5e9513fcc091bcf42e Jul 05 10:25:41.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-TestVolumeBootPattern-620052980 Jul 05 10:25:41.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-TestVolumeBootPattern-620052980-project Jul 05 10:25:41.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:25:41.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:25:41.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: c9203890ad7649f897b13ac7bdf6be3b Jul 05 10:25:41.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-TestVolumeBootPattern-620052980-project Jul 05 10:25:41.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:25:41.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:25:41.743572 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': '1eeacca5-4921-455f-80e9-acee90ae762f'} Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:25:41.744532 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:25:41.745918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:41.747032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6657933e-06d2-491e-8d06-04aa1ebc0305 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Returning 404 to user: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:25:41.747032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-6657933e-06d2-491e-8d06-04aa1ebc0305 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 404 len: 111 microversion: 2.1 time: 0.032260 Jul 05 10:25:41.747032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 523/1020] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:41 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:25:41.803119 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1091361366", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1298886845", "security_groups": [{"name": "tempest-secgroup-smoke-1344850936"}], "block_device_mapping_v2": [{"uuid": "da4c16ca-15da-4cce-a606-40d05ba2cc45", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "5a6b68d3-f3f2-400d-a2f3-b1cc33d83760"}]}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:25:41.916332 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.network.neutron [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] validate_networks() for [('5a6b68d3-f3f2-400d-a2f3-b1cc33d83760', None, None, None, None, None)] {{(pid=117812) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:25:42.294920 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-bd11a6f8-5f1f-4d80-ad61-daabf424dcd3 req-5b1b9b33-b7fd-492d-964a-ba44adfd8508 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1eeacca5-4921-455f-80e9-acee90ae762f", "name": "network-vif-deleted", "tag": "265f0d38-e515-44a1-95c5-3695adc2b78e"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:25:42.305358 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-bd11a6f8-5f1f-4d80-ad61-daabf424dcd3 req-5b1b9b33-b7fd-492d-964a-ba44adfd8508 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:42.305800 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-bd11a6f8-5f1f-4d80-ad61-daabf424dcd3 req-5b1b9b33-b7fd-492d-964a-ba44adfd8508 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:42.317490 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-bd11a6f8-5f1f-4d80-ad61-daabf424dcd3 req-5b1b9b33-b7fd-492d-964a-ba44adfd8508 service nova] Creating event network-vif-deleted:265f0d38-e515-44a1-95c5-3695adc2b78e for instance 1eeacca5-4921-455f-80e9-acee90ae762f on hv-cinder-maste Jul 05 10:25:42.323540 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-bd11a6f8-5f1f-4d80-ad61-daabf424dcd3 req-5b1b9b33-b7fd-492d-964a-ba44adfd8508 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037166 Jul 05 10:25:42.324255 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 524/1021] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:25:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:25:42.352698 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Flavor limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:25:42.353126 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Image limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:25:42.353602 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Flavor pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:25:42.354017 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Image pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:25:42.354435 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:25:42.363120 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:42.363575 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:42.364377 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:42.364850 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:42.382004 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Getting quotas for project 86eac45249ad4a5e9513fcc091bcf42e. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:25:42.386486 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Getting quotas for user c9203890ad7649f897b13ac7bdf6be3b and project 86eac45249ad4a5e9513fcc091bcf42e. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:25:42.441760 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Going to run 1 instances... {{(pid=117812) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:25:42.443873 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='da4c16ca-15da-4cce-a606-40d05ba2cc45',volume_size=None,volume_type=None)] {{(pid=117812) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:25:42.570639 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-adc5f92f-5ba8-4d4a-aea9-7fadcf367551 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.771398 Jul 05 10:25:42.571762 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 498/1022] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:25:41 2022] POST /compute/v2.1/servers => generated 406 bytes in 773 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:25:42.582573 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d0f19e76-6da5-401e-be26-80d9238f8afd tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:42.583331 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d0f19e76-6da5-401e-be26-80d9238f8afd tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:42.642235 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d0f19e76-6da5-401e-be26-80d9238f8afd tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:42.649376 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d0f19e76-6da5-401e-be26-80d9238f8afd tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1095 microversion: 2.1 time: 0.070998 Jul 05 10:25:42.649930 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 525/1023] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:42 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1095 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:25:43.370765 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e6a49572-da04-4929-a5d5-330112c35ca4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:43.371288 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e6a49572-da04-4929-a5d5-330112c35ca4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:43.377308 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e6a49572-da04-4929-a5d5-330112c35ca4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:43.377753 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e6a49572-da04-4929-a5d5-330112c35ca4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:43.378506 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e6a49572-da04-4929-a5d5-330112c35ca4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:43.378942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e6a49572-da04-4929-a5d5-330112c35ca4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:43.394755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-e6a49572-da04-4929-a5d5-330112c35ca4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] HTTP exception thrown: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e6a49572-da04-4929-a5d5-330112c35ca4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Request method failure captured: Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: DELETE /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a HTTP/1.1 Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA_xvpEELgE-eBUYHUhNsBeyr4iL6TX2JC5RBVSqxhiIRu0uvV5DpIroGwyHi1VHjPe6ULauFPG_08J8dk6bV7uqHLXSCeuk3phF15RwcRNPncEKU39B23DXNZT1IGuIE1R6RvECOKMp30Qx65b7eEKmzJ9mKE-_AEJ6raY_zH75ufY Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Openstack-Nova-Api-Version: 2.20 Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: f0c19d009f1b4ef09a40267bcf654107 Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:25:43.396467 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:25:43.398219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f0c19d009f1b4ef09a40267bcf654107"}]}] Jul 05 10:25:43.398219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:43.398219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: f0c19d009f1b4ef09a40267bcf654107 Jul 05 10:25:43.398219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:43.398219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-AttachVolumeShelveTestJSON-1813703734-project Jul 05 10:25:43.398219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:25:43.398219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:25:43.398219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: d4a608563189457dbcadf8d48acdd84a Jul 05 10:25:43.398219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1813703734-project Jul 05 10:25:43.398219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:25:43.398219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:43.398219 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': '1c90da36-1371-4861-9756-e275bf1f514a'} Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:25:43.399708 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:43.401346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:25:43.402879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e6a49572-da04-4929-a5d5-330112c35ca4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Returning 404 to user: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:25:43.402879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e6a49572-da04-4929-a5d5-330112c35ca4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "DELETE /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 404 len: 111 microversion: 2.20 time: 0.031462 Jul 05 10:25:43.402879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 499/1024] 10.222.0.20 () {60 vars in 1279 bytes} [Tue Jul 5 10:25:43 2022] DELETE /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 05 10:25:43.406981 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e2772f05-b964-4468-adb9-51ef9a532968 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:43.407596 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e2772f05-b964-4468-adb9-51ef9a532968 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: 1c90da36-1371-4861-9756-e275bf1f514a] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:43.414290 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e2772f05-b964-4468-adb9-51ef9a532968 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:43.414739 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e2772f05-b964-4468-adb9-51ef9a532968 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:43.415503 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e2772f05-b964-4468-adb9-51ef9a532968 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:43.416191 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e2772f05-b964-4468-adb9-51ef9a532968 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:43.431212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-e2772f05-b964-4468-adb9-51ef9a532968 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] HTTP exception thrown: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e2772f05-b964-4468-adb9-51ef9a532968 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Request method failure captured: Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a HTTP/1.1 Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA_xvpEELgE-eBUYHUhNsBeyr4iL6TX2JC5RBVSqxhiIRu0uvV5DpIroGwyHi1VHjPe6ULauFPG_08J8dk6bV7uqHLXSCeuk3phF15RwcRNPncEKU39B23DXNZT1IGuIE1R6RvECOKMp30Qx65b7eEKmzJ9mKE-_AEJ6raY_zH75ufY Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Openstack-Nova-Api-Version: 2.20 Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: f0c19d009f1b4ef09a40267bcf654107 Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:25:43.432888 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:25:43.434630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f0c19d009f1b4ef09a40267bcf654107"}]}] Jul 05 10:25:43.434630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:43.434630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: f0c19d009f1b4ef09a40267bcf654107 Jul 05 10:25:43.434630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:43.434630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeShelveTestJSON-1813703734-project Jul 05 10:25:43.434630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:25:43.434630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:25:43.434630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: d4a608563189457dbcadf8d48acdd84a Jul 05 10:25:43.434630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1813703734-project Jul 05 10:25:43.434630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:25:43.434630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:43.434630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': '1c90da36-1371-4861-9756-e275bf1f514a'} Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:25:43.435406 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. Jul 05 10:25:43.436736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:43.437835 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e2772f05-b964-4468-adb9-51ef9a532968 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Returning 404 to user: Instance 1c90da36-1371-4861-9756-e275bf1f514a could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:25:43.437835 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e2772f05-b964-4468-adb9-51ef9a532968 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a" status: 404 len: 111 microversion: 2.20 time: 0.031327 Jul 05 10:25:43.437835 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 526/1025] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:25:43 2022] GET /compute/v2.1/servers/1c90da36-1371-4861-9756-e275bf1f514a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 05 10:25:43.666271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-51508a3c-b0b2-45d9-bf5a-8a14cc040b84 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:43.666885 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-51508a3c-b0b2-45d9-bf5a-8a14cc040b84 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:43.672837 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-51508a3c-b0b2-45d9-bf5a-8a14cc040b84 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:43.673295 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-51508a3c-b0b2-45d9-bf5a-8a14cc040b84 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:43.674077 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-51508a3c-b0b2-45d9-bf5a-8a14cc040b84 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:43.674521 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-51508a3c-b0b2-45d9-bf5a-8a14cc040b84 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:43.758227 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-51508a3c-b0b2-45d9-bf5a-8a14cc040b84 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:43.765701 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-51508a3c-b0b2-45d9-bf5a-8a14cc040b84 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1211 microversion: 2.1 time: 0.103488 Jul 05 10:25:43.766284 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 500/1026] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:43 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1211 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:44.568942 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b82a729a-a701-4460-8730-96d267a4a027 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:44.569494 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b82a729a-a701-4460-8730-96d267a4a027 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:44.576403 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b82a729a-a701-4460-8730-96d267a4a027 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:44.576865 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b82a729a-a701-4460-8730-96d267a4a027 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:44.577685 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b82a729a-a701-4460-8730-96d267a4a027 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:44.578068 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b82a729a-a701-4460-8730-96d267a4a027 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:44.593047 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-b82a729a-a701-4460-8730-96d267a4a027 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] HTTP exception thrown: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b82a729a-a701-4460-8730-96d267a4a027 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Request method failure captured: Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: DELETE /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 HTTP/1.1 Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixA_xvpEELgE-eBUYHUhNsBeyr4iL6TX2JC5RBVSqxhiIRu0uvV5DpIroGwyHi1VHjPe6ULauFPG_08J8dk6bV7uqHLXSCeuk3phF15RwcRNPncEKU39B23DXNZT1IGuIE1R6RvECOKMp30Qx65b7eEKmzJ9mKE-_AEJ6raY_zH75ufY Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Openstack-Nova-Api-Version: 2.20 Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: f0c19d009f1b4ef09a40267bcf654107 Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:25:44.594600 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:25:44.596295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f0c19d009f1b4ef09a40267bcf654107"}]}] Jul 05 10:25:44.596295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:44.596295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: f0c19d009f1b4ef09a40267bcf654107 Jul 05 10:25:44.596295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:44.596295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeShelveTestJSON-1813703734-project Jul 05 10:25:44.596295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:25:44.596295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:25:44.596295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: d4a608563189457dbcadf8d48acdd84a Jul 05 10:25:44.596295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1813703734-project Jul 05 10:25:44.596295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:25:44.596295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:25:44.596295 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'ff442846-3ded-48c4-8a48-df125f46d0d6'} Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:25:44.597377 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:25:44.598938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:25:44.600892 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b82a729a-a701-4460-8730-96d267a4a027 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Returning 404 to user: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:25:44.600892 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b82a729a-a701-4460-8730-96d267a4a027 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "DELETE /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 404 len: 111 microversion: 2.20 time: 0.030883 Jul 05 10:25:44.600892 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 527/1027] 10.222.0.20 () {60 vars in 1279 bytes} [Tue Jul 5 10:25:44 2022] DELETE /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 05 10:25:44.604999 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-68b5ed31-6e68-4c27-9432-8a345128e1e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:44.605641 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-68b5ed31-6e68-4c27-9432-8a345128e1e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] [instance: ff442846-3ded-48c4-8a48-df125f46d0d6] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:44.612096 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-68b5ed31-6e68-4c27-9432-8a345128e1e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:44.612581 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-68b5ed31-6e68-4c27-9432-8a345128e1e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:44.613361 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-68b5ed31-6e68-4c27-9432-8a345128e1e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:44.613803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-68b5ed31-6e68-4c27-9432-8a345128e1e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:44.627375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-68b5ed31-6e68-4c27-9432-8a345128e1e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] HTTP exception thrown: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-68b5ed31-6e68-4c27-9432-8a345128e1e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Request method failure captured: Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 HTTP/1.1 Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixA_xvpEELgE-eBUYHUhNsBeyr4iL6TX2JC5RBVSqxhiIRu0uvV5DpIroGwyHi1VHjPe6ULauFPG_08J8dk6bV7uqHLXSCeuk3phF15RwcRNPncEKU39B23DXNZT1IGuIE1R6RvECOKMp30Qx65b7eEKmzJ9mKE-_AEJ6raY_zH75ufY Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Openstack-Nova-Api-Version: 2.20 Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: f0c19d009f1b4ef09a40267bcf654107 Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:25:44.628942 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:25:44.630607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f0c19d009f1b4ef09a40267bcf654107"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/f0c19d009f1b4ef09a40267bcf654107"}]}] Jul 05 10:25:44.630607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:44.630607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: f0c19d009f1b4ef09a40267bcf654107 Jul 05 10:25:44.630607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1813703734 Jul 05 10:25:44.630607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-AttachVolumeShelveTestJSON-1813703734-project Jul 05 10:25:44.630607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:25:44.630607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:25:44.630607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: d4a608563189457dbcadf8d48acdd84a Jul 05 10:25:44.630607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1813703734-project Jul 05 10:25:44.630607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:25:44.630607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:25:44.630607 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': 'ff442846-3ded-48c4-8a48-df125f46d0d6'} Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:25:44.631554 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. Jul 05 10:25:44.633542 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:25:44.634626 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-68b5ed31-6e68-4c27-9432-8a345128e1e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] Returning 404 to user: Instance ff442846-3ded-48c4-8a48-df125f46d0d6 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:25:44.634626 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-68b5ed31-6e68-4c27-9432-8a345128e1e4 tempest-AttachVolumeShelveTestJSON-1813703734 tempest-AttachVolumeShelveTestJSON-1813703734-project] 10.222.0.20 "GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6" status: 404 len: 111 microversion: 2.20 time: 0.029418 Jul 05 10:25:44.634626 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 501/1028] 10.222.0.20 () {60 vars in 1276 bytes} [Tue Jul 5 10:25:44 2022] GET /compute/v2.1/servers/ff442846-3ded-48c4-8a48-df125f46d0d6 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 05 10:25:44.782430 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a3f76c6b-a553-43d8-bd5e-160de0796eaf tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:44.782915 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a3f76c6b-a553-43d8-bd5e-160de0796eaf tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:44.790459 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a3f76c6b-a553-43d8-bd5e-160de0796eaf tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:44.791217 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a3f76c6b-a553-43d8-bd5e-160de0796eaf tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:44.791923 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a3f76c6b-a553-43d8-bd5e-160de0796eaf tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:44.792365 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a3f76c6b-a553-43d8-bd5e-160de0796eaf tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:45.028343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-a3f76c6b-a553-43d8-bd5e-160de0796eaf tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:45.036153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a3f76c6b-a553-43d8-bd5e-160de0796eaf tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1267 microversion: 2.1 time: 0.257761 Jul 05 10:25:45.041163 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 528/1029] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:44 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1267 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:45.115622 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-80c79274-b6fd-44b9-a986-1c3744c99475 req-2943ad38-e2cb-418d-815c-251c77e4a219 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b87058af-a15a-4a2b-9f05-e3f9b99e4a4d", "tag": "bc97aaf3-0c0d-415c-b3c3-645269085c58"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:25:45.128945 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-80c79274-b6fd-44b9-a986-1c3744c99475 req-2943ad38-e2cb-418d-815c-251c77e4a219 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:45.130267 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-80c79274-b6fd-44b9-a986-1c3744c99475 req-2943ad38-e2cb-418d-815c-251c77e4a219 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:45.150183 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-80c79274-b6fd-44b9-a986-1c3744c99475 req-2943ad38-e2cb-418d-815c-251c77e4a219 service nova] Creating event network-changed:bc97aaf3-0c0d-415c-b3c3-645269085c58 for instance b87058af-a15a-4a2b-9f05-e3f9b99e4a4d on hv-cinder-maste Jul 05 10:25:45.156662 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-80c79274-b6fd-44b9-a986-1c3744c99475 req-2943ad38-e2cb-418d-815c-251c77e4a219 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049005 Jul 05 10:25:45.157957 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 502/1030] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:25:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:25:46.055273 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-75fdd389-40df-4e69-b552-d86a33f25337 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:46.056121 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-75fdd389-40df-4e69-b552-d86a33f25337 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:46.062755 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-75fdd389-40df-4e69-b552-d86a33f25337 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:46.063280 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-75fdd389-40df-4e69-b552-d86a33f25337 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:46.064117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-75fdd389-40df-4e69-b552-d86a33f25337 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:46.064760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-75fdd389-40df-4e69-b552-d86a33f25337 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:46.195462 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-75fdd389-40df-4e69-b552-d86a33f25337 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:46.203130 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-75fdd389-40df-4e69-b552-d86a33f25337 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1267 microversion: 2.1 time: 0.152894 Jul 05 10:25:46.203951 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 529/1031] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:46 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:47.220981 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f2fac9f2-78ed-4302-be70-52e9cacc04b6 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:47.222648 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f2fac9f2-78ed-4302-be70-52e9cacc04b6 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:47.231042 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f2fac9f2-78ed-4302-be70-52e9cacc04b6 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:47.231566 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f2fac9f2-78ed-4302-be70-52e9cacc04b6 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:47.232426 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f2fac9f2-78ed-4302-be70-52e9cacc04b6 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:47.232970 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f2fac9f2-78ed-4302-be70-52e9cacc04b6 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:47.462233 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f2fac9f2-78ed-4302-be70-52e9cacc04b6 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:47.471617 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f2fac9f2-78ed-4302-be70-52e9cacc04b6 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1267 microversion: 2.1 time: 0.255454 Jul 05 10:25:47.472208 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 503/1032] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:47 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1267 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:48.488188 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-86c6211e-9087-44a7-a82f-6fa4e67a1fd3 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:48.488845 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-86c6211e-9087-44a7-a82f-6fa4e67a1fd3 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:48.495174 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-86c6211e-9087-44a7-a82f-6fa4e67a1fd3 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:48.495628 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-86c6211e-9087-44a7-a82f-6fa4e67a1fd3 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:48.496416 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-86c6211e-9087-44a7-a82f-6fa4e67a1fd3 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:48.496861 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-86c6211e-9087-44a7-a82f-6fa4e67a1fd3 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:48.655045 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-86c6211e-9087-44a7-a82f-6fa4e67a1fd3 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:48.662563 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-86c6211e-9087-44a7-a82f-6fa4e67a1fd3 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1267 microversion: 2.1 time: 0.178197 Jul 05 10:25:48.663826 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 530/1033] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:48 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1267 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:49.679638 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7856bc37-3d3a-4d00-8f5c-5a07e85cc347 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:49.680339 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7856bc37-3d3a-4d00-8f5c-5a07e85cc347 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:49.686985 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7856bc37-3d3a-4d00-8f5c-5a07e85cc347 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:49.687432 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7856bc37-3d3a-4d00-8f5c-5a07e85cc347 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:49.688193 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7856bc37-3d3a-4d00-8f5c-5a07e85cc347 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:49.688654 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7856bc37-3d3a-4d00-8f5c-5a07e85cc347 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:49.912933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-7856bc37-3d3a-4d00-8f5c-5a07e85cc347 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:49.921031 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7856bc37-3d3a-4d00-8f5c-5a07e85cc347 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1267 microversion: 2.1 time: 0.245512 Jul 05 10:25:49.931412 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 504/1034] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:49 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1267 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:50.940285 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-697ea4f6-411e-4489-825c-999f80a2f952 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:50.941402 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-697ea4f6-411e-4489-825c-999f80a2f952 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:50.950372 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-697ea4f6-411e-4489-825c-999f80a2f952 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:50.951395 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-697ea4f6-411e-4489-825c-999f80a2f952 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:50.952222 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-697ea4f6-411e-4489-825c-999f80a2f952 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:50.952758 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-697ea4f6-411e-4489-825c-999f80a2f952 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:51.110366 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-697ea4f6-411e-4489-825c-999f80a2f952 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:51.119126 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-697ea4f6-411e-4489-825c-999f80a2f952 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1267 microversion: 2.1 time: 0.183430 Jul 05 10:25:51.119974 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 531/1035] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:50 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1267 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:52.138488 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-44fb0f22-259b-41ee-a198-54bc746a8395 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:52.140278 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-44fb0f22-259b-41ee-a198-54bc746a8395 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:52.148079 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-44fb0f22-259b-41ee-a198-54bc746a8395 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:52.148748 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-44fb0f22-259b-41ee-a198-54bc746a8395 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:52.149699 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-44fb0f22-259b-41ee-a198-54bc746a8395 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:52.150300 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-44fb0f22-259b-41ee-a198-54bc746a8395 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:52.322545 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-44fb0f22-259b-41ee-a198-54bc746a8395 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:52.331427 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-44fb0f22-259b-41ee-a198-54bc746a8395 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1267 microversion: 2.1 time: 0.197008 Jul 05 10:25:52.333116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 505/1036] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:52 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1267 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:53.349460 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5efd8e76-0851-4a4a-91c2-336de4b6e564 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:53.351353 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5efd8e76-0851-4a4a-91c2-336de4b6e564 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:53.358698 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:25:53.360614 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5efd8e76-0851-4a4a-91c2-336de4b6e564 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:53.361171 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5efd8e76-0851-4a4a-91c2-336de4b6e564 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:53.362154 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5efd8e76-0851-4a4a-91c2-336de4b6e564 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:53.362670 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5efd8e76-0851-4a4a-91c2-336de4b6e564 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:53.528861 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5efd8e76-0851-4a4a-91c2-336de4b6e564 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:53.538085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5efd8e76-0851-4a4a-91c2-336de4b6e564 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1267 microversion: 2.1 time: 0.192170 Jul 05 10:25:53.544853 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 532/1037] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:53 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1267 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:25:53.707299 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:25:53.708119 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=6 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:25:53.819416 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-405b9449-d4e3-4320-b11c-728711d1465d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Action: 'create', calling method: .version_select at 0x7feb896a21f0>, body: {"keypair": {"name": "tempest-keypair-1739685357", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMyVCKzL7k1YglodJg/7eIrbNmbY6xuf8SPOy6/mSaLdaVxSpLRg5JtyWGkrJOf2KYSIj71tQcfXXOpFSyx0PU1++fXuQa6BIiX7rHtr3XZ5FeFilFhl1VRFF9OHNLg1Zg=="}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:25:53.833352 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-405b9449-d4e3-4320-b11c-728711d1465d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Getting quotas for project 8b5c262b91db42cb9e4042b180f9fb4d. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:25:53.837173 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-405b9449-d4e3-4320-b11c-728711d1465d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Getting quotas for user b131286ad9144575a13972e25cab20db and project 8b5c262b91db42cb9e4042b180f9fb4d. Resources: {'key_pairs'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:25:53.860113 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-405b9449-d4e3-4320-b11c-728711d1465d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.157320 Jul 05 10:25:53.861130 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 506/1038] 10.222.0.20 () {60 vars in 1158 bytes} [Tue Jul 5 10:25:53 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:25:54.556373 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1bdd4df9-6e06-443e-8aed-4d1b311cb78f tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:54.557040 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1bdd4df9-6e06-443e-8aed-4d1b311cb78f tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:54.564332 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1bdd4df9-6e06-443e-8aed-4d1b311cb78f tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:54.564814 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1bdd4df9-6e06-443e-8aed-4d1b311cb78f tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:54.565586 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1bdd4df9-6e06-443e-8aed-4d1b311cb78f tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:54.566018 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1bdd4df9-6e06-443e-8aed-4d1b311cb78f tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:54.769613 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1bdd4df9-6e06-443e-8aed-4d1b311cb78f tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:54.790041 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1bdd4df9-6e06-443e-8aed-4d1b311cb78f tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1267 microversion: 2.1 time: 0.238101 Jul 05 10:25:54.791490 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 533/1039] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:54 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1267 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:55.048919 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-7e655b79-8cf0-441e-bcb1-5991685f2a1f req-8ea15c96-8c5e-4923-b59c-8a4df030da03 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b87058af-a15a-4a2b-9f05-e3f9b99e4a4d", "name": "network-vif-plugged", "status": "completed", "tag": "bc97aaf3-0c0d-415c-b3c3-645269085c58"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:25:55.060079 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-7e655b79-8cf0-441e-bcb1-5991685f2a1f req-8ea15c96-8c5e-4923-b59c-8a4df030da03 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:55.061116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-7e655b79-8cf0-441e-bcb1-5991685f2a1f req-8ea15c96-8c5e-4923-b59c-8a4df030da03 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:55.075522 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-7e655b79-8cf0-441e-bcb1-5991685f2a1f req-8ea15c96-8c5e-4923-b59c-8a4df030da03 service nova] Creating event network-vif-plugged:bc97aaf3-0c0d-415c-b3c3-645269085c58 for instance b87058af-a15a-4a2b-9f05-e3f9b99e4a4d on hv-cinder-maste Jul 05 10:25:55.082666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-7e655b79-8cf0-441e-bcb1-5991685f2a1f req-8ea15c96-8c5e-4923-b59c-8a4df030da03 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037294 Jul 05 10:25:55.083285 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 507/1040] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:25:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:25:55.266530 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-460128535", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "905e65c9-d2c9-40b0-a3db-38b3be82f3f2"}], "security_groups": [{"name": "tempest-securitygroup--752043838"}], "key_name": "tempest-keypair-1739685357", "user_data": "}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:25:55.405006 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.network.neutron [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] validate_networks() for [('905e65c9-d2c9-40b0-a3db-38b3be82f3f2', None, None, None, None, None)] {{(pid=117813) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:25:55.807652 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-accbf03c-19bf-41ed-9498-cda3a97a77b9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:55.807834 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Flavor limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:25:55.808268 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Image limits 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:25:55.808364 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-accbf03c-19bf-41ed-9498-cda3a97a77b9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:55.808810 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Flavor pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:25:55.809212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Image pref 0:0:0 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:25:55.809746 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.virt.hardware [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117813) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:25:55.815025 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-accbf03c-19bf-41ed-9498-cda3a97a77b9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:55.815477 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-accbf03c-19bf-41ed-9498-cda3a97a77b9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:55.816245 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-accbf03c-19bf-41ed-9498-cda3a97a77b9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:55.816716 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-accbf03c-19bf-41ed-9498-cda3a97a77b9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:55.819131 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:55.819610 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:55.820500 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:55.821128 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:55.838263 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Getting quotas for project 8b5c262b91db42cb9e4042b180f9fb4d. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:25:55.843389 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Getting quotas for user b131286ad9144575a13972e25cab20db and project 8b5c262b91db42cb9e4042b180f9fb4d. Resources: {'cores', 'ram', 'instances'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:25:55.855502 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Going to run 1 instances... {{(pid=117813) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:25:55.857797 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117813) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:25:55.909101 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-9afe6c68-d988-4e19-ab11-fcd7b5b15f99 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.648230 Jul 05 10:25:55.910478 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 534/1041] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:25:55 2022] POST /compute/v2.1/servers => generated 401 bytes in 650 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:25:55.920211 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-0d479b96-8a90-4140-9b40-1248a84dee28 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:55.921407 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-0d479b96-8a90-4140-9b40-1248a84dee28 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:55.992824 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-0d479b96-8a90-4140-9b40-1248a84dee28 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:55.998087 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-accbf03c-19bf-41ed-9498-cda3a97a77b9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:56.001110 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-0d479b96-8a90-4140-9b40-1248a84dee28 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1237 microversion: 2.1 time: 0.084719 Jul 05 10:25:56.002146 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 535/1042] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:55 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1237 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:25:56.006642 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-accbf03c-19bf-41ed-9498-cda3a97a77b9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1267 microversion: 2.1 time: 0.203179 Jul 05 10:25:56.007205 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 508/1043] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:55 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1267 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:25:57.023846 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b69a530d-3f94-4636-9bcf-869d3b2848a9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:57.024592 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b69a530d-3f94-4636-9bcf-869d3b2848a9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:57.029017 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2e222220-1710-49af-ba73-73a497bdd290 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:57.030199 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2e222220-1710-49af-ba73-73a497bdd290 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:57.038576 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b69a530d-3f94-4636-9bcf-869d3b2848a9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:57.041556 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2e222220-1710-49af-ba73-73a497bdd290 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:57.042194 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2e222220-1710-49af-ba73-73a497bdd290 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:57.043154 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2e222220-1710-49af-ba73-73a497bdd290 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:57.043750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2e222220-1710-49af-ba73-73a497bdd290 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:57.044507 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b69a530d-3f94-4636-9bcf-869d3b2848a9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:57.048440 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b69a530d-3f94-4636-9bcf-869d3b2848a9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:57.049059 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b69a530d-3f94-4636-9bcf-869d3b2848a9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:57.084573 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:25:57.192618 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2e222220-1710-49af-ba73-73a497bdd290 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:57.213089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2e222220-1710-49af-ba73-73a497bdd290 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1289 microversion: 2.1 time: 0.197877 Jul 05 10:25:57.214226 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 536/1044] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:57 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1289 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:57.326122 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b69a530d-3f94-4636-9bcf-869d3b2848a9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:57.335148 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b69a530d-3f94-4636-9bcf-869d3b2848a9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1454 microversion: 2.1 time: 0.315682 Jul 05 10:25:57.337250 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 509/1045] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:57 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1454 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:57.354471 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c1c966db-7309-4125-af6b-dfa64c7a0903 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:57.355359 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c1c966db-7309-4125-af6b-dfa64c7a0903 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:57.362344 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c1c966db-7309-4125-af6b-dfa64c7a0903 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:57.363055 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c1c966db-7309-4125-af6b-dfa64c7a0903 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:57.364064 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c1c966db-7309-4125-af6b-dfa64c7a0903 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:57.364775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c1c966db-7309-4125-af6b-dfa64c7a0903 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:57.610802 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c1c966db-7309-4125-af6b-dfa64c7a0903 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:57.619976 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c1c966db-7309-4125-af6b-dfa64c7a0903 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1454 microversion: 2.1 time: 0.268796 Jul 05 10:25:57.622028 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 537/1046] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:57 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1454 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:25:58.230990 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d50f923f-bf28-4ce3-ba9d-34c44c2c251e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:58.232358 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d50f923f-bf28-4ce3-ba9d-34c44c2c251e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:58.245341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d50f923f-bf28-4ce3-ba9d-34c44c2c251e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:58.246025 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d50f923f-bf28-4ce3-ba9d-34c44c2c251e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:58.247054 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d50f923f-bf28-4ce3-ba9d-34c44c2c251e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:58.247734 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d50f923f-bf28-4ce3-ba9d-34c44c2c251e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:58.505622 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d50f923f-bf28-4ce3-ba9d-34c44c2c251e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:58.510797 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:25:58.519054 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d50f923f-bf28-4ce3-ba9d-34c44c2c251e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1362 microversion: 2.1 time: 0.291831 Jul 05 10:25:58.520014 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 510/1047] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:58 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1362 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 05 10:25:59.095617 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-4fa3cd8f-5a57-483d-a65d-9dcb5b956e9e req-7afe1ab5-263c-49c7-b7e4-fa9167031078 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a65a032c-5b68-4e65-b02c-d76e6547c26c", "tag": "b6c06b3b-2cbc-4544-9d94-78a4a3d5a141"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:25:59.112701 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-4fa3cd8f-5a57-483d-a65d-9dcb5b956e9e req-7afe1ab5-263c-49c7-b7e4-fa9167031078 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:59.114145 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-4fa3cd8f-5a57-483d-a65d-9dcb5b956e9e req-7afe1ab5-263c-49c7-b7e4-fa9167031078 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:59.137763 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-4fa3cd8f-5a57-483d-a65d-9dcb5b956e9e req-7afe1ab5-263c-49c7-b7e4-fa9167031078 service nova] Creating event network-changed:b6c06b3b-2cbc-4544-9d94-78a4a3d5a141 for instance a65a032c-5b68-4e65-b02c-d76e6547c26c on hv-cinder-maste Jul 05 10:25:59.149080 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-4fa3cd8f-5a57-483d-a65d-9dcb5b956e9e req-7afe1ab5-263c-49c7-b7e4-fa9167031078 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056717 Jul 05 10:25:59.150299 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 538/1048] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:25:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:25:59.464269 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-99586026-b85d-424a-9bc6-0496d6f4c923 req-6dc8ec11-7289-4761-9273-3af4ef0fe8b4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b87058af-a15a-4a2b-9f05-e3f9b99e4a4d", "tag": "bc97aaf3-0c0d-415c-b3c3-645269085c58"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:25:59.472299 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-99586026-b85d-424a-9bc6-0496d6f4c923 req-6dc8ec11-7289-4761-9273-3af4ef0fe8b4 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:59.473035 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-99586026-b85d-424a-9bc6-0496d6f4c923 req-6dc8ec11-7289-4761-9273-3af4ef0fe8b4 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:59.484616 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-99586026-b85d-424a-9bc6-0496d6f4c923 req-6dc8ec11-7289-4761-9273-3af4ef0fe8b4 service nova] Creating event network-changed:bc97aaf3-0c0d-415c-b3c3-645269085c58 for instance b87058af-a15a-4a2b-9f05-e3f9b99e4a4d on hv-cinder-maste Jul 05 10:25:59.490559 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-99586026-b85d-424a-9bc6-0496d6f4c923 req-6dc8ec11-7289-4761-9273-3af4ef0fe8b4 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.098940 Jul 05 10:25:59.491735 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 511/1049] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:25:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:25:59.538476 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c6d870c1-7bb8-4b4d-918d-5b20bfc21dd1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:25:59.539177 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c6d870c1-7bb8-4b4d-918d-5b20bfc21dd1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:25:59.547136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c6d870c1-7bb8-4b4d-918d-5b20bfc21dd1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:59.547587 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c6d870c1-7bb8-4b4d-918d-5b20bfc21dd1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:59.548369 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c6d870c1-7bb8-4b4d-918d-5b20bfc21dd1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:25:59.550338 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c6d870c1-7bb8-4b4d-918d-5b20bfc21dd1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:25:59.785520 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-c6d870c1-7bb8-4b4d-918d-5b20bfc21dd1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:25:59.793080 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c6d870c1-7bb8-4b4d-918d-5b20bfc21dd1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1362 microversion: 2.1 time: 0.259169 Jul 05 10:25:59.793664 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 539/1050] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:25:59 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1362 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:26:00.811064 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d2837f85-0974-4fed-bd4b-8ffaa40bc80b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:26:00.811839 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d2837f85-0974-4fed-bd4b-8ffaa40bc80b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:26:00.818919 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d2837f85-0974-4fed-bd4b-8ffaa40bc80b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:00.819374 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d2837f85-0974-4fed-bd4b-8ffaa40bc80b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:00.820171 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d2837f85-0974-4fed-bd4b-8ffaa40bc80b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:00.820645 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d2837f85-0974-4fed-bd4b-8ffaa40bc80b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:00.954216 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d2837f85-0974-4fed-bd4b-8ffaa40bc80b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:26:00.962117 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d2837f85-0974-4fed-bd4b-8ffaa40bc80b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1362 microversion: 2.1 time: 0.155798 Jul 05 10:26:00.962684 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 512/1051] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:26:00 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1362 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:26:01.979222 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2c2b75f3-76a1-465f-8da1-bf529b47285f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:26:01.979820 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2c2b75f3-76a1-465f-8da1-bf529b47285f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:26:01.986403 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2c2b75f3-76a1-465f-8da1-bf529b47285f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:01.986844 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2c2b75f3-76a1-465f-8da1-bf529b47285f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:01.987609 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2c2b75f3-76a1-465f-8da1-bf529b47285f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:01.988038 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2c2b75f3-76a1-465f-8da1-bf529b47285f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:02.309883 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2c2b75f3-76a1-465f-8da1-bf529b47285f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:26:02.317978 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2c2b75f3-76a1-465f-8da1-bf529b47285f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1362 microversion: 2.1 time: 0.342876 Jul 05 10:26:02.318630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 540/1052] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:26:01 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1362 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:26:03.334825 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e1b105ec-178b-4b55-b14e-ad0fdab331a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:26:03.335436 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e1b105ec-178b-4b55-b14e-ad0fdab331a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:26:03.341884 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e1b105ec-178b-4b55-b14e-ad0fdab331a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:03.342328 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e1b105ec-178b-4b55-b14e-ad0fdab331a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:03.343085 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e1b105ec-178b-4b55-b14e-ad0fdab331a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:03.343522 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e1b105ec-178b-4b55-b14e-ad0fdab331a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:03.471271 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e1b105ec-178b-4b55-b14e-ad0fdab331a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:26:03.479258 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e1b105ec-178b-4b55-b14e-ad0fdab331a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1362 microversion: 2.1 time: 0.148328 Jul 05 10:26:03.479909 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 513/1053] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:26:03 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1362 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:26:04.495708 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-60921afe-96c0-4139-8dd7-6fcc915b83ba tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:26:04.496312 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-60921afe-96c0-4139-8dd7-6fcc915b83ba tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:26:04.503561 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-60921afe-96c0-4139-8dd7-6fcc915b83ba tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:04.504033 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-60921afe-96c0-4139-8dd7-6fcc915b83ba tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:04.504805 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-60921afe-96c0-4139-8dd7-6fcc915b83ba tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:04.505239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-60921afe-96c0-4139-8dd7-6fcc915b83ba tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:04.632542 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-60921afe-96c0-4139-8dd7-6fcc915b83ba tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:26:04.639876 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-60921afe-96c0-4139-8dd7-6fcc915b83ba tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1362 microversion: 2.1 time: 0.148032 Jul 05 10:26:04.640456 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 541/1054] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:26:04 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1362 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:26:05.028986 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-32669f3f-b0fc-4c00-a833-7034a45e3d76 req-65198b70-fa7b-4b4f-95ff-dff338331ad8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a65a032c-5b68-4e65-b02c-d76e6547c26c", "name": "network-vif-plugged", "status": "completed", "tag": "b6c06b3b-2cbc-4544-9d94-78a4a3d5a141"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:26:05.032556 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=4 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:26:05.038919 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-32669f3f-b0fc-4c00-a833-7034a45e3d76 req-65198b70-fa7b-4b4f-95ff-dff338331ad8 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:05.039367 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-32669f3f-b0fc-4c00-a833-7034a45e3d76 req-65198b70-fa7b-4b4f-95ff-dff338331ad8 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:05.050777 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-32669f3f-b0fc-4c00-a833-7034a45e3d76 req-65198b70-fa7b-4b4f-95ff-dff338331ad8 service nova] Creating event network-vif-plugged:b6c06b3b-2cbc-4544-9d94-78a4a3d5a141 for instance a65a032c-5b68-4e65-b02c-d76e6547c26c on hv-cinder-maste Jul 05 10:26:05.055822 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-32669f3f-b0fc-4c00-a833-7034a45e3d76 req-65198b70-fa7b-4b4f-95ff-dff338331ad8 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031026 Jul 05 10:26:05.056439 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 514/1055] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:26:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:26:05.656170 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8864655c-17ef-46dd-9317-a9bb1b693164 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:26:05.656796 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8864655c-17ef-46dd-9317-a9bb1b693164 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:26:05.659192 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=77,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:26:05.665109 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8864655c-17ef-46dd-9317-a9bb1b693164 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:05.665555 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8864655c-17ef-46dd-9317-a9bb1b693164 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:05.666354 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8864655c-17ef-46dd-9317-a9bb1b693164 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:05.666863 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8864655c-17ef-46dd-9317-a9bb1b693164 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:05.795088 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8864655c-17ef-46dd-9317-a9bb1b693164 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:26:05.802997 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8864655c-17ef-46dd-9317-a9bb1b693164 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1362 microversion: 2.1 time: 0.150692 Jul 05 10:26:05.803558 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 542/1056] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:26:05 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1362 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:26:06.821841 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-bf66f622-92fc-4045-8ac8-ba06e70ea021 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:26:06.822454 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-bf66f622-92fc-4045-8ac8-ba06e70ea021 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:26:06.824446 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=79,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:26:06.830460 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bf66f622-92fc-4045-8ac8-ba06e70ea021 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:06.830923 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bf66f622-92fc-4045-8ac8-ba06e70ea021 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:06.831696 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bf66f622-92fc-4045-8ac8-ba06e70ea021 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:06.832140 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-bf66f622-92fc-4045-8ac8-ba06e70ea021 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:06.963816 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-bf66f622-92fc-4045-8ac8-ba06e70ea021 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:26:06.971190 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-bf66f622-92fc-4045-8ac8-ba06e70ea021 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1362 microversion: 2.1 time: 0.154879 Jul 05 10:26:06.971753 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 515/1057] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:26:06 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1362 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:26:07.987873 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-65d12b00-7ecf-46f0-83ce-5d8a78707f61 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:26:07.988512 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-65d12b00-7ecf-46f0-83ce-5d8a78707f61 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:26:07.990869 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:26:07.991537 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=8 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:26:07.997967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-65d12b00-7ecf-46f0-83ce-5d8a78707f61 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:07.998415 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-65d12b00-7ecf-46f0-83ce-5d8a78707f61 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:07.999176 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-65d12b00-7ecf-46f0-83ce-5d8a78707f61 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:07.999610 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-65d12b00-7ecf-46f0-83ce-5d8a78707f61 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:08.126655 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-65d12b00-7ecf-46f0-83ce-5d8a78707f61 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:26:08.134735 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-65d12b00-7ecf-46f0-83ce-5d8a78707f61 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1548 microversion: 2.1 time: 0.150776 Jul 05 10:26:08.135305 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 543/1058] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:26:07 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1548 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:26:08.150733 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7cdcb295-7189-4d5a-970a-ca305afa6a48 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:26:08.151221 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7cdcb295-7189-4d5a-970a-ca305afa6a48 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:26:08.157613 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7cdcb295-7189-4d5a-970a-ca305afa6a48 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:08.158066 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7cdcb295-7189-4d5a-970a-ca305afa6a48 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:08.158835 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7cdcb295-7189-4d5a-970a-ca305afa6a48 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:08.159278 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7cdcb295-7189-4d5a-970a-ca305afa6a48 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:08.234938 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7cdcb295-7189-4d5a-970a-ca305afa6a48 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.087967 Jul 05 10:26:08.235523 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 516/1059] 10.222.0.20 () {58 vars in 1274 bytes} [Tue Jul 5 10:26:08 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/os-interface => generated 284 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:26:09.400136 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-ffa3c4b4-b4aa-4fc9-a088-e4a60f19fe99 req-79b5ea56-dfcc-4935-bd5c-d0b0c2d3bf23 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a65a032c-5b68-4e65-b02c-d76e6547c26c", "tag": "b6c06b3b-2cbc-4544-9d94-78a4a3d5a141"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:26:09.408639 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-ffa3c4b4-b4aa-4fc9-a088-e4a60f19fe99 req-79b5ea56-dfcc-4935-bd5c-d0b0c2d3bf23 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:26:09.409087 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-ffa3c4b4-b4aa-4fc9-a088-e4a60f19fe99 req-79b5ea56-dfcc-4935-bd5c-d0b0c2d3bf23 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:26:09.420544 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-ffa3c4b4-b4aa-4fc9-a088-e4a60f19fe99 req-79b5ea56-dfcc-4935-bd5c-d0b0c2d3bf23 service nova] Creating event network-changed:b6c06b3b-2cbc-4544-9d94-78a4a3d5a141 for instance a65a032c-5b68-4e65-b02c-d76e6547c26c on hv-cinder-maste Jul 05 10:26:09.426194 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-ffa3c4b4-b4aa-4fc9-a088-e4a60f19fe99 req-79b5ea56-dfcc-4935-bd5c-d0b0c2d3bf23 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029459 Jul 05 10:26:09.426815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 544/1060] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:26:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:27:24.427621 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=3 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:24.428476 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=26 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:24.432697 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5572874d-72d6-4f2c-9bce-b1a09986d8db tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:24.433180 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5572874d-72d6-4f2c-9bce-b1a09986d8db tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:24.439755 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5572874d-72d6-4f2c-9bce-b1a09986d8db tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:24.440201 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5572874d-72d6-4f2c-9bce-b1a09986d8db tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:24.440990 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5572874d-72d6-4f2c-9bce-b1a09986d8db tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:24.441427 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5572874d-72d6-4f2c-9bce-b1a09986d8db tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:24.464081 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5572874d-72d6-4f2c-9bce-b1a09986d8db tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/ips" status: 200 len: 146 microversion: 2.1 time: 0.038357 Jul 05 10:27:24.464662 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 517/1061] 10.222.0.20 () {58 vars in 1247 bytes} [Tue Jul 5 10:27:24 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/ips => generated 146 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:27:26.416840 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=5 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:26.417569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=43 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:27:26.422303 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-c162c203-5b2b-4604-a5c6-9e7ed38b394b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c9694535-2dc7-4168-91e5-34935d4e544d"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:27:26.423431 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-c162c203-5b2b-4604-a5c6-9e7ed38b394b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:26.430246 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c162c203-5b2b-4604-a5c6-9e7ed38b394b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:26.430691 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c162c203-5b2b-4604-a5c6-9e7ed38b394b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:26.431446 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c162c203-5b2b-4604-a5c6-9e7ed38b394b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:26.431880 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-c162c203-5b2b-4604-a5c6-9e7ed38b394b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:26.679252 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-c162c203-5b2b-4604-a5c6-9e7ed38b394b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "POST /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.264225 Jul 05 10:27:26.679870 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 545/1062] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:27:26 2022] POST /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/os-volume_attachments => generated 194 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:27:34.153097 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-23128be7-e0c0-4847-9209-8de38d59f303 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:27:34.159910 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-23128be7-e0c0-4847-9209-8de38d59f303 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:34.160358 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-23128be7-e0c0-4847-9209-8de38d59f303 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:34.217889 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-23128be7-e0c0-4847-9209-8de38d59f303 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Going to try to stop instance {{(pid=117812) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 05 10:27:34.260024 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-23128be7-e0c0-4847-9209-8de38d59f303 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "POST /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/action" status: 202 len: 0 microversion: 2.1 time: 0.110739 Jul 05 10:27:34.260664 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 518/1063] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:27:34 2022] POST /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/action => generated 0 bytes in 112 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:27:34.270284 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e9d48be9-6f65-42de-af23-bfff4a1c5c49 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:34.270974 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e9d48be9-6f65-42de-af23-bfff4a1c5c49 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:34.277568 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e9d48be9-6f65-42de-af23-bfff4a1c5c49 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:34.278043 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e9d48be9-6f65-42de-af23-bfff4a1c5c49 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:34.278782 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e9d48be9-6f65-42de-af23-bfff4a1c5c49 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:34.279217 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e9d48be9-6f65-42de-af23-bfff4a1c5c49 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:34.409251 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e9d48be9-6f65-42de-af23-bfff4a1c5c49 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:34.417018 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e9d48be9-6f65-42de-af23-bfff4a1c5c49 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.151980 Jul 05 10:27:34.417840 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 546/1064] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:34 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:35.434803 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ea36e671-18de-425a-ae3f-d0a466ac2836 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:35.435500 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ea36e671-18de-425a-ae3f-d0a466ac2836 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:35.442321 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ea36e671-18de-425a-ae3f-d0a466ac2836 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:35.442766 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ea36e671-18de-425a-ae3f-d0a466ac2836 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:35.443525 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ea36e671-18de-425a-ae3f-d0a466ac2836 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:35.443961 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ea36e671-18de-425a-ae3f-d0a466ac2836 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:35.569872 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-ea36e671-18de-425a-ae3f-d0a466ac2836 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:35.577657 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ea36e671-18de-425a-ae3f-d0a466ac2836 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.147095 Jul 05 10:27:35.578210 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 519/1065] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:35 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:36.595591 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-37435733-bc20-4e34-9291-96df245c8fe8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:36.596200 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-37435733-bc20-4e34-9291-96df245c8fe8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:36.602797 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-37435733-bc20-4e34-9291-96df245c8fe8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:36.603246 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-37435733-bc20-4e34-9291-96df245c8fe8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:36.604012 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-37435733-bc20-4e34-9291-96df245c8fe8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:36.604475 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-37435733-bc20-4e34-9291-96df245c8fe8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:36.735189 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-37435733-bc20-4e34-9291-96df245c8fe8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:36.743033 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-37435733-bc20-4e34-9291-96df245c8fe8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.151960 Jul 05 10:27:36.743602 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 547/1066] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:36 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:37.760320 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0add7782-669c-453a-9799-250fccd707e9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:37.760993 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0add7782-669c-453a-9799-250fccd707e9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:37.767944 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0add7782-669c-453a-9799-250fccd707e9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:37.768412 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0add7782-669c-453a-9799-250fccd707e9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:37.769185 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0add7782-669c-453a-9799-250fccd707e9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:37.769626 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0add7782-669c-453a-9799-250fccd707e9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:37.898204 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-0add7782-669c-453a-9799-250fccd707e9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:37.906057 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0add7782-669c-453a-9799-250fccd707e9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.149587 Jul 05 10:27:37.906617 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 520/1067] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:37 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:38.923278 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1e510531-3dd5-4445-942a-26f26d58ead0 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:38.924764 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1e510531-3dd5-4445-942a-26f26d58ead0 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:38.931318 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1e510531-3dd5-4445-942a-26f26d58ead0 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:38.931774 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1e510531-3dd5-4445-942a-26f26d58ead0 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:38.932587 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1e510531-3dd5-4445-942a-26f26d58ead0 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:38.933033 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1e510531-3dd5-4445-942a-26f26d58ead0 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:39.061019 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1e510531-3dd5-4445-942a-26f26d58ead0 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:39.068951 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1e510531-3dd5-4445-942a-26f26d58ead0 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.149573 Jul 05 10:27:39.069516 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 548/1068] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:38 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:40.085832 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-4fe92660-9360-4ec4-b379-e9d6df8c91f4 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:40.086444 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-4fe92660-9360-4ec4-b379-e9d6df8c91f4 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:40.093002 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4fe92660-9360-4ec4-b379-e9d6df8c91f4 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:40.093489 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4fe92660-9360-4ec4-b379-e9d6df8c91f4 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:40.094310 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4fe92660-9360-4ec4-b379-e9d6df8c91f4 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:40.094737 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4fe92660-9360-4ec4-b379-e9d6df8c91f4 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:40.221620 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-4fe92660-9360-4ec4-b379-e9d6df8c91f4 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:40.229536 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-4fe92660-9360-4ec4-b379-e9d6df8c91f4 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.147593 Jul 05 10:27:40.230102 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 521/1069] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:40 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:41.248371 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-dc4b82cf-b5ee-4a72-b626-6e18267e3f27 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:41.249146 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-dc4b82cf-b5ee-4a72-b626-6e18267e3f27 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:41.256817 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dc4b82cf-b5ee-4a72-b626-6e18267e3f27 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:41.257268 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dc4b82cf-b5ee-4a72-b626-6e18267e3f27 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:41.260575 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dc4b82cf-b5ee-4a72-b626-6e18267e3f27 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:41.260575 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-dc4b82cf-b5ee-4a72-b626-6e18267e3f27 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:41.383812 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-dc4b82cf-b5ee-4a72-b626-6e18267e3f27 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:41.392200 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-dc4b82cf-b5ee-4a72-b626-6e18267e3f27 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.148124 Jul 05 10:27:41.392786 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 549/1070] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:41 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:42.409181 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8292ad90-ca1e-4636-b62e-5edf4770a57c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:42.409787 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-8292ad90-ca1e-4636-b62e-5edf4770a57c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:42.416475 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8292ad90-ca1e-4636-b62e-5edf4770a57c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:42.417190 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8292ad90-ca1e-4636-b62e-5edf4770a57c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:42.417958 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8292ad90-ca1e-4636-b62e-5edf4770a57c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:42.418399 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8292ad90-ca1e-4636-b62e-5edf4770a57c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:42.431084 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:27:42.431566 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Tue Jul 5 10:27:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:27:42.547581 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8292ad90-ca1e-4636-b62e-5edf4770a57c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:42.555460 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-8292ad90-ca1e-4636-b62e-5edf4770a57c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.150146 Jul 05 10:27:42.556025 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 522/1071] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:42 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:43.573193 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5f2c878a-b284-4601-9afc-b17b255b04b1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:43.573723 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5f2c878a-b284-4601-9afc-b17b255b04b1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:43.580783 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5f2c878a-b284-4601-9afc-b17b255b04b1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:43.581228 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5f2c878a-b284-4601-9afc-b17b255b04b1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:43.581978 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5f2c878a-b284-4601-9afc-b17b255b04b1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:43.582403 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5f2c878a-b284-4601-9afc-b17b255b04b1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:43.710999 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-5f2c878a-b284-4601-9afc-b17b255b04b1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:43.718864 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5f2c878a-b284-4601-9afc-b17b255b04b1 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.149736 Jul 05 10:27:43.719422 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 550/1072] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:43 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:44.735768 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-2cb9591f-86f3-4cdc-be26-4b63ed15d208 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:44.736379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-2cb9591f-86f3-4cdc-be26-4b63ed15d208 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:44.742869 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2cb9591f-86f3-4cdc-be26-4b63ed15d208 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:44.743320 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2cb9591f-86f3-4cdc-be26-4b63ed15d208 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:44.744085 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2cb9591f-86f3-4cdc-be26-4b63ed15d208 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:44.744559 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-2cb9591f-86f3-4cdc-be26-4b63ed15d208 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:44.872575 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-2cb9591f-86f3-4cdc-be26-4b63ed15d208 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:44.881143 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-2cb9591f-86f3-4cdc-be26-4b63ed15d208 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.149219 Jul 05 10:27:44.881707 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 523/1073] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:44 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:45.898585 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-fd13787f-9df8-445f-9179-afec7c7cdcc6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:45.899171 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-fd13787f-9df8-445f-9179-afec7c7cdcc6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:45.905583 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fd13787f-9df8-445f-9179-afec7c7cdcc6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:45.906552 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fd13787f-9df8-445f-9179-afec7c7cdcc6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:45.907300 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fd13787f-9df8-445f-9179-afec7c7cdcc6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:45.907727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fd13787f-9df8-445f-9179-afec7c7cdcc6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:46.033959 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-fd13787f-9df8-445f-9179-afec7c7cdcc6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:46.041995 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-fd13787f-9df8-445f-9179-afec7c7cdcc6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.147087 Jul 05 10:27:46.042547 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 551/1074] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:45 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:47.058514 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-408e463e-6848-4689-94fc-ca5b28e047b6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:47.059150 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-408e463e-6848-4689-94fc-ca5b28e047b6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:47.066641 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-408e463e-6848-4689-94fc-ca5b28e047b6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:47.067076 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-408e463e-6848-4689-94fc-ca5b28e047b6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:47.067994 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-408e463e-6848-4689-94fc-ca5b28e047b6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:47.068455 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-408e463e-6848-4689-94fc-ca5b28e047b6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:47.199427 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-408e463e-6848-4689-94fc-ca5b28e047b6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:47.207642 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-408e463e-6848-4689-94fc-ca5b28e047b6 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.152984 Jul 05 10:27:47.208221 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 524/1075] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:47 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:48.224707 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-099f5c0a-bae8-4c83-9e8d-4b28b6b5e9d3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:48.225315 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-099f5c0a-bae8-4c83-9e8d-4b28b6b5e9d3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:48.231918 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-099f5c0a-bae8-4c83-9e8d-4b28b6b5e9d3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:48.232367 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-099f5c0a-bae8-4c83-9e8d-4b28b6b5e9d3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:48.233160 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-099f5c0a-bae8-4c83-9e8d-4b28b6b5e9d3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:48.233598 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-099f5c0a-bae8-4c83-9e8d-4b28b6b5e9d3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:48.360026 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-099f5c0a-bae8-4c83-9e8d-4b28b6b5e9d3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:48.367890 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-099f5c0a-bae8-4c83-9e8d-4b28b6b5e9d3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.147118 Jul 05 10:27:48.368471 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 552/1076] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:48 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:49.385490 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5386f6dd-044c-40a1-b207-cd15c0346724 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:49.386111 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5386f6dd-044c-40a1-b207-cd15c0346724 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:49.393211 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5386f6dd-044c-40a1-b207-cd15c0346724 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:49.393661 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5386f6dd-044c-40a1-b207-cd15c0346724 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:49.394425 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5386f6dd-044c-40a1-b207-cd15c0346724 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:49.394865 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5386f6dd-044c-40a1-b207-cd15c0346724 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:49.519483 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5386f6dd-044c-40a1-b207-cd15c0346724 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:49.527907 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5386f6dd-044c-40a1-b207-cd15c0346724 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1723 microversion: 2.1 time: 0.146237 Jul 05 10:27:49.528441 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 525/1077] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:49 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1723 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:50.544960 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e8cf5781-91a5-4f1d-ae81-09ea54ae99a5 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:50.545567 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e8cf5781-91a5-4f1d-ae81-09ea54ae99a5 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:50.551986 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e8cf5781-91a5-4f1d-ae81-09ea54ae99a5 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:50.552473 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e8cf5781-91a5-4f1d-ae81-09ea54ae99a5 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:50.553235 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e8cf5781-91a5-4f1d-ae81-09ea54ae99a5 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:50.553672 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e8cf5781-91a5-4f1d-ae81-09ea54ae99a5 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:50.681925 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-e8cf5781-91a5-4f1d-ae81-09ea54ae99a5 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:50.689778 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e8cf5781-91a5-4f1d-ae81-09ea54ae99a5 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1700 microversion: 2.1 time: 0.148859 Jul 05 10:27:50.690343 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 553/1078] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:50 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1700 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:50.706338 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f869da29-d391-47f3-a840-558a958e333f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:27:50.712823 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f869da29-d391-47f3-a840-558a958e333f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:50.713266 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f869da29-d391-47f3-a840-558a958e333f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:50.736883 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f869da29-d391-47f3-a840-558a958e333f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Going to try to start instance {{(pid=117812) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 05 10:27:50.776570 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f869da29-d391-47f3-a840-558a958e333f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "POST /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/action" status: 202 len: 0 microversion: 2.1 time: 0.074382 Jul 05 10:27:50.777637 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 526/1079] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:27:50 2022] POST /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/action => generated 0 bytes in 76 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:27:50.786139 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1625c834-e300-483a-b465-09ec70be4c8d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:50.786747 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1625c834-e300-483a-b465-09ec70be4c8d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:50.793032 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1625c834-e300-483a-b465-09ec70be4c8d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:50.793555 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1625c834-e300-483a-b465-09ec70be4c8d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:50.794341 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1625c834-e300-483a-b465-09ec70be4c8d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:50.794812 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1625c834-e300-483a-b465-09ec70be4c8d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:50.950908 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-1625c834-e300-483a-b465-09ec70be4c8d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:50.959464 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1625c834-e300-483a-b465-09ec70be4c8d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1709 microversion: 2.1 time: 0.177139 Jul 05 10:27:50.960023 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 554/1080] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:50 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1709 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:51.977260 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-632da09c-f03c-4c18-a0ff-8300638362d8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:51.977953 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-632da09c-f03c-4c18-a0ff-8300638362d8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:51.985163 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-632da09c-f03c-4c18-a0ff-8300638362d8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:51.985646 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-632da09c-f03c-4c18-a0ff-8300638362d8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:51.986413 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-632da09c-f03c-4c18-a0ff-8300638362d8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:51.986857 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-632da09c-f03c-4c18-a0ff-8300638362d8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:52.119182 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-632da09c-f03c-4c18-a0ff-8300638362d8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:52.127615 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-632da09c-f03c-4c18-a0ff-8300638362d8 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1709 microversion: 2.1 time: 0.154554 Jul 05 10:27:52.128177 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 527/1081] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:51 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1709 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:27:53.144969 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-048ae248-7579-4254-8c01-78583e5dcbbf tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:27:53.145628 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-048ae248-7579-4254-8c01-78583e5dcbbf tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:27:53.151953 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-048ae248-7579-4254-8c01-78583e5dcbbf tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:53.152412 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-048ae248-7579-4254-8c01-78583e5dcbbf tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:53.153174 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-048ae248-7579-4254-8c01-78583e5dcbbf tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:27:53.153633 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-048ae248-7579-4254-8c01-78583e5dcbbf tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:27:53.279422 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-048ae248-7579-4254-8c01-78583e5dcbbf tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:27:53.287367 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-048ae248-7579-4254-8c01-78583e5dcbbf tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1713 microversion: 2.1 time: 0.146624 Jul 05 10:27:53.287917 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 555/1082] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:27:53 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1713 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:03.887531 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-304040890", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1298886845", "security_groups": [{"name": "tempest-secgroup-smoke-1344850936"}], "block_device_mapping_v2": [{"uuid": "5fba2647-3a7b-49da-a3fe-e6728cd9c52c", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "5a6b68d3-f3f2-400d-a2f3-b1cc33d83760"}]}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:28:03.896426 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=22 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:03.897072 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=72,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:04.000644 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.network.neutron [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] validate_networks() for [('5a6b68d3-f3f2-400d-a2f3-b1cc33d83760', None, None, None, None, None)] {{(pid=117812) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:28:04.358414 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Flavor limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:28:04.358832 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Image limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:28:04.359303 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Flavor pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:28:04.359711 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Image pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:28:04.360124 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:28:04.368887 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:04.369337 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:04.370132 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:04.370570 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:04.387383 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Getting quotas for project 86eac45249ad4a5e9513fcc091bcf42e. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:28:04.391364 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Getting quotas for user c9203890ad7649f897b13ac7bdf6be3b and project 86eac45249ad4a5e9513fcc091bcf42e. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:28:04.446463 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Going to run 1 instances... {{(pid=117812) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:28:04.448566 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='5fba2647-3a7b-49da-a3fe-e6728cd9c52c',volume_size=None,volume_type=None)] {{(pid=117812) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:28:04.572658 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ae45365c-6af1-409b-b312-0683e4705385 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.688893 Jul 05 10:28:04.573339 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 528/1083] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:28:03 2022] POST /compute/v2.1/servers => generated 406 bytes in 690 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:28:04.583731 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d52572ce-a813-4315-ad05-ad73a0616e15 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:04.584350 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d52572ce-a813-4315-ad05-ad73a0616e15 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:04.587412 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=22 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:04.588499 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=71,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:04.643644 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d52572ce-a813-4315-ad05-ad73a0616e15 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:04.650371 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d52572ce-a813-4315-ad05-ad73a0616e15 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1094 microversion: 2.1 time: 0.070866 Jul 05 10:28:04.651125 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 556/1084] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:04 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1094 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:05.668425 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-aedcd290-c488-4ecc-bd9e-1f3c0a093e0b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:05.669045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-aedcd290-c488-4ecc-bd9e-1f3c0a093e0b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:05.676661 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-aedcd290-c488-4ecc-bd9e-1f3c0a093e0b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:05.677115 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-aedcd290-c488-4ecc-bd9e-1f3c0a093e0b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:05.677876 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-aedcd290-c488-4ecc-bd9e-1f3c0a093e0b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:05.678323 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-aedcd290-c488-4ecc-bd9e-1f3c0a093e0b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:05.751392 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-aedcd290-c488-4ecc-bd9e-1f3c0a093e0b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:05.758996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-aedcd290-c488-4ecc-bd9e-1f3c0a093e0b tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1200 microversion: 2.1 time: 0.094697 Jul 05 10:28:05.759553 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 529/1085] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:05 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1200 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:06.777058 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-4a9e768a-e90b-4fc4-848d-200e2d05e8a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:06.777667 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-4a9e768a-e90b-4fc4-848d-200e2d05e8a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:06.784148 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4a9e768a-e90b-4fc4-848d-200e2d05e8a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:06.784579 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4a9e768a-e90b-4fc4-848d-200e2d05e8a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:06.785345 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4a9e768a-e90b-4fc4-848d-200e2d05e8a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:06.785780 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4a9e768a-e90b-4fc4-848d-200e2d05e8a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:07.010173 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-4a9e768a-e90b-4fc4-848d-200e2d05e8a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:07.017733 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-4a9e768a-e90b-4fc4-848d-200e2d05e8a4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1266 microversion: 2.1 time: 0.244548 Jul 05 10:28:07.018512 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 557/1086] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:06 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1266 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:07.468838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-cb55f049-f0ee-4cb1-aec0-98df876ecfa9 req-8cad1aaa-f2b0-4fc7-83b6-15379032f22e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e91854f0-736f-436d-ad34-11e66f585c5d", "tag": "dc432f06-5b7d-409d-a315-06582b118d3d"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:28:07.477220 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-cb55f049-f0ee-4cb1-aec0-98df876ecfa9 req-8cad1aaa-f2b0-4fc7-83b6-15379032f22e service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:07.477668 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-cb55f049-f0ee-4cb1-aec0-98df876ecfa9 req-8cad1aaa-f2b0-4fc7-83b6-15379032f22e service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:07.490659 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-cb55f049-f0ee-4cb1-aec0-98df876ecfa9 req-8cad1aaa-f2b0-4fc7-83b6-15379032f22e service nova] Creating event network-changed:dc432f06-5b7d-409d-a315-06582b118d3d for instance e91854f0-736f-436d-ad34-11e66f585c5d on hv-cinder-maste Jul 05 10:28:07.496223 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-cb55f049-f0ee-4cb1-aec0-98df876ecfa9 req-8cad1aaa-f2b0-4fc7-83b6-15379032f22e service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.092500 Jul 05 10:28:07.497482 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 530/1087] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:28:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:28:08.034990 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-db2495d8-91a9-4438-93f4-068da628d0a1 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:08.035587 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-db2495d8-91a9-4438-93f4-068da628d0a1 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:08.042270 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-db2495d8-91a9-4438-93f4-068da628d0a1 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:08.042712 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-db2495d8-91a9-4438-93f4-068da628d0a1 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:08.043467 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-db2495d8-91a9-4438-93f4-068da628d0a1 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:08.043901 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-db2495d8-91a9-4438-93f4-068da628d0a1 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:08.175727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-db2495d8-91a9-4438-93f4-068da628d0a1 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:08.183105 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-db2495d8-91a9-4438-93f4-068da628d0a1 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1266 microversion: 2.1 time: 0.151929 Jul 05 10:28:08.184013 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 558/1088] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:08 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1266 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:09.201813 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-8bd60177-597a-4baf-8e38-90dfe99f72db tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:09.202525 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-8bd60177-597a-4baf-8e38-90dfe99f72db tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:09.209397 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8bd60177-597a-4baf-8e38-90dfe99f72db tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:09.209853 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8bd60177-597a-4baf-8e38-90dfe99f72db tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:09.210629 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8bd60177-597a-4baf-8e38-90dfe99f72db tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:09.211068 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-8bd60177-597a-4baf-8e38-90dfe99f72db tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:09.357546 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-8bd60177-597a-4baf-8e38-90dfe99f72db tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:09.365158 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-8bd60177-597a-4baf-8e38-90dfe99f72db tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1266 microversion: 2.1 time: 0.168049 Jul 05 10:28:09.365723 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 531/1089] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:09 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1266 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:10.381703 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8e5ed28e-ce72-4caf-bce3-f71a45145c94 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:10.382307 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8e5ed28e-ce72-4caf-bce3-f71a45145c94 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:10.389380 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8e5ed28e-ce72-4caf-bce3-f71a45145c94 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:10.389827 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8e5ed28e-ce72-4caf-bce3-f71a45145c94 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:10.390580 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8e5ed28e-ce72-4caf-bce3-f71a45145c94 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:10.391017 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8e5ed28e-ce72-4caf-bce3-f71a45145c94 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:10.518907 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8e5ed28e-ce72-4caf-bce3-f71a45145c94 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:10.526287 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8e5ed28e-ce72-4caf-bce3-f71a45145c94 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1266 microversion: 2.1 time: 0.148458 Jul 05 10:28:10.526852 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 559/1090] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:10 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1266 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:11.543101 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d31b776d-0bda-4609-8afc-67313845a37d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:11.543662 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d31b776d-0bda-4609-8afc-67313845a37d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:11.550811 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d31b776d-0bda-4609-8afc-67313845a37d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:11.551259 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d31b776d-0bda-4609-8afc-67313845a37d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:11.552021 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d31b776d-0bda-4609-8afc-67313845a37d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:11.552479 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d31b776d-0bda-4609-8afc-67313845a37d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:11.683429 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d31b776d-0bda-4609-8afc-67313845a37d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:11.690891 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d31b776d-0bda-4609-8afc-67313845a37d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1266 microversion: 2.1 time: 0.151966 Jul 05 10:28:11.691459 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 532/1091] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:11 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1266 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:12.707500 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-08e02031-bcd8-4007-acb6-3cacfadfb311 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:12.708083 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-08e02031-bcd8-4007-acb6-3cacfadfb311 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:12.714470 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-08e02031-bcd8-4007-acb6-3cacfadfb311 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:12.714913 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-08e02031-bcd8-4007-acb6-3cacfadfb311 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:12.716171 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-08e02031-bcd8-4007-acb6-3cacfadfb311 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:12.716629 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-08e02031-bcd8-4007-acb6-3cacfadfb311 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:12.844450 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-08e02031-bcd8-4007-acb6-3cacfadfb311 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:12.851736 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-08e02031-bcd8-4007-acb6-3cacfadfb311 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1266 microversion: 2.1 time: 0.148101 Jul 05 10:28:12.852296 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 560/1092] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:12 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1266 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:13.868889 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-02ba15f5-4e1b-41f7-9dfa-ac9f985f56cd tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:13.869418 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-02ba15f5-4e1b-41f7-9dfa-ac9f985f56cd tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:13.876402 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-02ba15f5-4e1b-41f7-9dfa-ac9f985f56cd tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:13.876860 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-02ba15f5-4e1b-41f7-9dfa-ac9f985f56cd tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:13.877623 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-02ba15f5-4e1b-41f7-9dfa-ac9f985f56cd tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:13.878057 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-02ba15f5-4e1b-41f7-9dfa-ac9f985f56cd tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:14.011662 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-02ba15f5-4e1b-41f7-9dfa-ac9f985f56cd tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:14.019050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-02ba15f5-4e1b-41f7-9dfa-ac9f985f56cd tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1266 microversion: 2.1 time: 0.154065 Jul 05 10:28:14.019605 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 533/1093] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:13 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:15.035969 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d2a68f2b-f2d9-4098-b931-100f96d4a99e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:15.036621 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d2a68f2b-f2d9-4098-b931-100f96d4a99e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:15.038931 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:15.044556 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d2a68f2b-f2d9-4098-b931-100f96d4a99e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:15.045008 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d2a68f2b-f2d9-4098-b931-100f96d4a99e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:15.045796 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d2a68f2b-f2d9-4098-b931-100f96d4a99e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:15.046234 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d2a68f2b-f2d9-4098-b931-100f96d4a99e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:15.175445 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d2a68f2b-f2d9-4098-b931-100f96d4a99e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:15.182789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d2a68f2b-f2d9-4098-b931-100f96d4a99e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1266 microversion: 2.1 time: 0.150656 Jul 05 10:28:15.183347 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 561/1094] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:15 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1266 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:16.198940 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-f0f5f3bb-f859-4687-91d1-d3bbc18654e4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:16.199557 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-f0f5f3bb-f859-4687-91d1-d3bbc18654e4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:16.201586 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:16.202237 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=6 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:16.208775 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f0f5f3bb-f859-4687-91d1-d3bbc18654e4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:16.209226 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f0f5f3bb-f859-4687-91d1-d3bbc18654e4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:16.209994 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f0f5f3bb-f859-4687-91d1-d3bbc18654e4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:16.210437 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-f0f5f3bb-f859-4687-91d1-d3bbc18654e4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:16.345272 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-f0f5f3bb-f859-4687-91d1-d3bbc18654e4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:16.352926 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-f0f5f3bb-f859-4687-91d1-d3bbc18654e4 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1266 microversion: 2.1 time: 0.157720 Jul 05 10:28:16.353507 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 534/1095] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:16 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:17.369768 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3ea39a84-200c-4f61-a568-355a7f07354a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:17.370379 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3ea39a84-200c-4f61-a568-355a7f07354a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:17.377289 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3ea39a84-200c-4f61-a568-355a7f07354a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:17.377731 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3ea39a84-200c-4f61-a568-355a7f07354a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:17.378487 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3ea39a84-200c-4f61-a568-355a7f07354a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:17.378916 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3ea39a84-200c-4f61-a568-355a7f07354a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:17.510789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-3ea39a84-200c-4f61-a568-355a7f07354a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:17.518249 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3ea39a84-200c-4f61-a568-355a7f07354a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1266 microversion: 2.1 time: 0.152716 Jul 05 10:28:17.518903 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 562/1096] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:17 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:18.534880 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-d60f0997-7932-42c2-8196-3ce79163ab45 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:18.535457 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-d60f0997-7932-42c2-8196-3ce79163ab45 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:18.542127 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d60f0997-7932-42c2-8196-3ce79163ab45 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:18.542574 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d60f0997-7932-42c2-8196-3ce79163ab45 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:18.543333 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d60f0997-7932-42c2-8196-3ce79163ab45 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:18.543763 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-d60f0997-7932-42c2-8196-3ce79163ab45 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:18.675258 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-d60f0997-7932-42c2-8196-3ce79163ab45 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:18.682612 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-d60f0997-7932-42c2-8196-3ce79163ab45 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1266 microversion: 2.1 time: 0.151671 Jul 05 10:28:18.683173 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 535/1097] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:18 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1266 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:19.695544 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-91d90f5c-12e8-4a69-89c8-08cf0440e504 req-ae4d78d3-9948-4498-b8eb-29d042a74201 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e91854f0-736f-436d-ad34-11e66f585c5d", "name": "network-vif-plugged", "status": "completed", "tag": "dc432f06-5b7d-409d-a315-06582b118d3d"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:28:19.699132 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-6bb445be-e93d-4962-8d77-e57d387da406 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:19.700267 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-6bb445be-e93d-4962-8d77-e57d387da406 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:19.704515 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-91d90f5c-12e8-4a69-89c8-08cf0440e504 req-ae4d78d3-9948-4498-b8eb-29d042a74201 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:19.705028 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-91d90f5c-12e8-4a69-89c8-08cf0440e504 req-ae4d78d3-9948-4498-b8eb-29d042a74201 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:19.706691 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6bb445be-e93d-4962-8d77-e57d387da406 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:19.707139 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6bb445be-e93d-4962-8d77-e57d387da406 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:19.707896 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6bb445be-e93d-4962-8d77-e57d387da406 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:19.708334 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-6bb445be-e93d-4962-8d77-e57d387da406 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:19.719320 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-91d90f5c-12e8-4a69-89c8-08cf0440e504 req-ae4d78d3-9948-4498-b8eb-29d042a74201 service nova] Creating event network-vif-plugged:dc432f06-5b7d-409d-a315-06582b118d3d for instance e91854f0-736f-436d-ad34-11e66f585c5d on hv-cinder-maste Jul 05 10:28:19.724459 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-91d90f5c-12e8-4a69-89c8-08cf0440e504 req-ae4d78d3-9948-4498-b8eb-29d042a74201 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093410 Jul 05 10:28:19.725055 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 563/1098] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:28:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:28:19.845414 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-6bb445be-e93d-4962-8d77-e57d387da406 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:19.852806 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-6bb445be-e93d-4962-8d77-e57d387da406 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1266 microversion: 2.1 time: 0.157562 Jul 05 10:28:19.853302 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 536/1099] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:19 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:20.870783 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-49a1bf90-024d-440a-b9c1-02665f2095c7 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:20.871471 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-49a1bf90-024d-440a-b9c1-02665f2095c7 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:20.877711 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-49a1bf90-024d-440a-b9c1-02665f2095c7 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:20.878154 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-49a1bf90-024d-440a-b9c1-02665f2095c7 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:20.878907 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-49a1bf90-024d-440a-b9c1-02665f2095c7 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:20.879342 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-49a1bf90-024d-440a-b9c1-02665f2095c7 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:21.011467 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-49a1bf90-024d-440a-b9c1-02665f2095c7 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:21.018809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-49a1bf90-024d-440a-b9c1-02665f2095c7 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1266 microversion: 2.1 time: 0.152701 Jul 05 10:28:21.019366 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 564/1100] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:20 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1266 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:22.035653 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c6d66c87-52bf-4f36-bfe6-888fdb209c76 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:22.036313 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-c6d66c87-52bf-4f36-bfe6-888fdb209c76 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:22.042968 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c6d66c87-52bf-4f36-bfe6-888fdb209c76 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:22.043417 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c6d66c87-52bf-4f36-bfe6-888fdb209c76 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:22.044183 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c6d66c87-52bf-4f36-bfe6-888fdb209c76 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:22.044644 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c6d66c87-52bf-4f36-bfe6-888fdb209c76 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:22.172574 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-c6d66c87-52bf-4f36-bfe6-888fdb209c76 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:22.179953 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-c6d66c87-52bf-4f36-bfe6-888fdb209c76 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1453 microversion: 2.1 time: 0.148188 Jul 05 10:28:22.180559 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 537/1101] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:22 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1453 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:22.195483 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2f48b975-e59d-4945-9485-0e120f79031d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:22.196090 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2f48b975-e59d-4945-9485-0e120f79031d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:22.202978 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f48b975-e59d-4945-9485-0e120f79031d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:22.203426 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f48b975-e59d-4945-9485-0e120f79031d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:22.204187 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f48b975-e59d-4945-9485-0e120f79031d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:22.204658 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2f48b975-e59d-4945-9485-0e120f79031d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:22.331181 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-2f48b975-e59d-4945-9485-0e120f79031d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:22.338615 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2f48b975-e59d-4945-9485-0e120f79031d tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1453 microversion: 2.1 time: 0.146891 Jul 05 10:28:22.339177 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 565/1102] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:22 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1453 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:23.973785 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-2d515593-31d1-4ead-a213-b48e0d32ec5f req-ca4cda0d-a502-4a9f-9e61-c99b4905ced2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e91854f0-736f-436d-ad34-11e66f585c5d", "tag": "dc432f06-5b7d-409d-a315-06582b118d3d"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:28:23.982321 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-2d515593-31d1-4ead-a213-b48e0d32ec5f req-ca4cda0d-a502-4a9f-9e61-c99b4905ced2 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:23.982771 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-2d515593-31d1-4ead-a213-b48e0d32ec5f req-ca4cda0d-a502-4a9f-9e61-c99b4905ced2 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:23.995424 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-2d515593-31d1-4ead-a213-b48e0d32ec5f req-ca4cda0d-a502-4a9f-9e61-c99b4905ced2 service nova] Creating event network-changed:dc432f06-5b7d-409d-a315-06582b118d3d for instance e91854f0-736f-436d-ad34-11e66f585c5d on hv-cinder-maste Jul 05 10:28:24.001376 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-2d515593-31d1-4ead-a213-b48e0d32ec5f req-ca4cda0d-a502-4a9f-9e61-c99b4905ced2 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031272 Jul 05 10:28:24.002446 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 538/1103] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:28:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:28:36.407424 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:28:36.407870 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Tue Jul 5 10:28:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:28:40.588221 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:28:40.588719 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Tue Jul 5 10:28:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:28:57.007036 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5814490b-2332-463c-b492-2d4402e04edc tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:57.007557 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5814490b-2332-463c-b492-2d4402e04edc tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:57.009818 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=22 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:57.010608 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=50 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:57.016048 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5814490b-2332-463c-b492-2d4402e04edc tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:57.016537 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5814490b-2332-463c-b492-2d4402e04edc tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:57.017298 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5814490b-2332-463c-b492-2d4402e04edc tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:57.017734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5814490b-2332-463c-b492-2d4402e04edc tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:57.096005 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.objects.instance [None req-5814490b-2332-463c-b492-2d4402e04edc tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lazy-loading 'services' on Instance uuid a65a032c-5b68-4e65-b02c-d76e6547c26c {{(pid=117813) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:28:57.166327 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5814490b-2332-463c-b492-2d4402e04edc tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/os-volume_attachments/c9694535-2dc7-4168-91e5-34935d4e544d" status: 202 len: 0 microversion: 2.1 time: 0.163041 Jul 05 10:28:57.166922 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 566/1104] 10.222.0.20 () {58 vars in 1415 bytes} [Tue Jul 5 10:28:57 2022] DELETE /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/os-volume_attachments/c9694535-2dc7-4168-91e5-34935d4e544d => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:28:58.282761 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=35,nova_api:INSERT=3 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:58.283505 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=54 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:28:58.288627 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-be4eb121-f48b-4b23-85be-cd4777b1daaa tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:28:58.297422 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be4eb121-f48b-4b23-85be-cd4777b1daaa tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:58.297879 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-be4eb121-f48b-4b23-85be-cd4777b1daaa tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:58.320592 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-be4eb121-f48b-4b23-85be-cd4777b1daaa tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Going to try to stop instance {{(pid=117812) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 05 10:28:58.361045 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-be4eb121-f48b-4b23-85be-cd4777b1daaa tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "POST /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/action" status: 202 len: 0 microversion: 2.1 time: 0.079798 Jul 05 10:28:58.361659 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 539/1105] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:28:58 2022] POST /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/action => generated 0 bytes in 81 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:28:58.370882 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b3ec7c2c-709a-4b71-8fff-52cd00e3332d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:58.371565 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b3ec7c2c-709a-4b71-8fff-52cd00e3332d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:58.377848 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b3ec7c2c-709a-4b71-8fff-52cd00e3332d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:58.378293 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b3ec7c2c-709a-4b71-8fff-52cd00e3332d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:58.379058 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b3ec7c2c-709a-4b71-8fff-52cd00e3332d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:58.379522 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b3ec7c2c-709a-4b71-8fff-52cd00e3332d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:58.512955 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b3ec7c2c-709a-4b71-8fff-52cd00e3332d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:58.520338 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b3ec7c2c-709a-4b71-8fff-52cd00e3332d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1677 microversion: 2.1 time: 0.154205 Jul 05 10:28:58.520947 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 567/1106] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:58 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1677 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:28:59.539298 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a5eacaed-df56-4ba5-8eaa-cdc099c13fcd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:28:59.540008 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-a5eacaed-df56-4ba5-8eaa-cdc099c13fcd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:28:59.546713 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5eacaed-df56-4ba5-8eaa-cdc099c13fcd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:59.547170 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5eacaed-df56-4ba5-8eaa-cdc099c13fcd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:59.547936 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5eacaed-df56-4ba5-8eaa-cdc099c13fcd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:28:59.548375 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-a5eacaed-df56-4ba5-8eaa-cdc099c13fcd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:28:59.677782 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-a5eacaed-df56-4ba5-8eaa-cdc099c13fcd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:28:59.685743 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a5eacaed-df56-4ba5-8eaa-cdc099c13fcd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1677 microversion: 2.1 time: 0.151385 Jul 05 10:28:59.686305 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 540/1107] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:28:59 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1677 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:29:00.703336 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-fcb1208b-1dda-4154-bc26-444c1e9bdcde tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:29:00.703946 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-fcb1208b-1dda-4154-bc26-444c1e9bdcde tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:29:00.710389 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fcb1208b-1dda-4154-bc26-444c1e9bdcde tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:00.710836 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fcb1208b-1dda-4154-bc26-444c1e9bdcde tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:00.711587 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fcb1208b-1dda-4154-bc26-444c1e9bdcde tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:00.712035 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-fcb1208b-1dda-4154-bc26-444c1e9bdcde tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:00.838416 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-fcb1208b-1dda-4154-bc26-444c1e9bdcde tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:29:00.845863 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-fcb1208b-1dda-4154-bc26-444c1e9bdcde tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1677 microversion: 2.1 time: 0.147024 Jul 05 10:29:00.846426 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 568/1108] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:29:00 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1677 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:29:01.863037 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-84c48c7d-867f-40ae-899f-f1e1c087453c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:29:01.863635 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-84c48c7d-867f-40ae-899f-f1e1c087453c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:29:01.869797 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-84c48c7d-867f-40ae-899f-f1e1c087453c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:01.870277 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-84c48c7d-867f-40ae-899f-f1e1c087453c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:01.871114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-84c48c7d-867f-40ae-899f-f1e1c087453c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:01.872114 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-84c48c7d-867f-40ae-899f-f1e1c087453c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:02.000513 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-84c48c7d-867f-40ae-899f-f1e1c087453c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:29:02.007838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-84c48c7d-867f-40ae-899f-f1e1c087453c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1677 microversion: 2.1 time: 0.148666 Jul 05 10:29:02.008401 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 541/1109] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:29:01 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1677 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:29:03.024683 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-cadeeae8-ebf1-47f0-9a5a-219f34ea2db7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:29:03.025363 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-cadeeae8-ebf1-47f0-9a5a-219f34ea2db7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:29:03.032328 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cadeeae8-ebf1-47f0-9a5a-219f34ea2db7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:03.032808 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cadeeae8-ebf1-47f0-9a5a-219f34ea2db7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:03.033570 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cadeeae8-ebf1-47f0-9a5a-219f34ea2db7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:03.034008 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-cadeeae8-ebf1-47f0-9a5a-219f34ea2db7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:03.165227 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-cadeeae8-ebf1-47f0-9a5a-219f34ea2db7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:29:03.172774 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-cadeeae8-ebf1-47f0-9a5a-219f34ea2db7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1677 microversion: 2.1 time: 0.151936 Jul 05 10:29:03.173335 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 569/1110] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:29:03 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1677 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:29:04.189638 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5619754d-ad07-48f0-85bb-94932f715a2f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:29:04.190260 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5619754d-ad07-48f0-85bb-94932f715a2f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:29:04.196758 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5619754d-ad07-48f0-85bb-94932f715a2f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:04.197211 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5619754d-ad07-48f0-85bb-94932f715a2f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:04.197983 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5619754d-ad07-48f0-85bb-94932f715a2f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:04.198425 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5619754d-ad07-48f0-85bb-94932f715a2f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:04.566527 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5619754d-ad07-48f0-85bb-94932f715a2f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:29:04.573988 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5619754d-ad07-48f0-85bb-94932f715a2f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1677 microversion: 2.1 time: 0.388219 Jul 05 10:29:04.575189 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 542/1111] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:29:04 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1677 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:29:05.591777 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-59c42a31-1cfc-4530-951c-a3b03d0f3a26 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:29:05.592309 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-59c42a31-1cfc-4530-951c-a3b03d0f3a26 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:29:05.598971 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-59c42a31-1cfc-4530-951c-a3b03d0f3a26 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:05.599418 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-59c42a31-1cfc-4530-951c-a3b03d0f3a26 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:05.600178 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-59c42a31-1cfc-4530-951c-a3b03d0f3a26 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:05.600639 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-59c42a31-1cfc-4530-951c-a3b03d0f3a26 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:05.725006 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-59c42a31-1cfc-4530-951c-a3b03d0f3a26 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:29:05.732857 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-59c42a31-1cfc-4530-951c-a3b03d0f3a26 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1677 microversion: 2.1 time: 0.145073 Jul 05 10:29:05.733419 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 570/1112] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:29:05 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1677 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:29:06.750369 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-b935a785-8e87-45d0-8915-a7bb457dbc98 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:29:06.750983 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-b935a785-8e87-45d0-8915-a7bb457dbc98 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:29:06.757507 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b935a785-8e87-45d0-8915-a7bb457dbc98 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:06.757962 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b935a785-8e87-45d0-8915-a7bb457dbc98 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:06.758733 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b935a785-8e87-45d0-8915-a7bb457dbc98 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:06.759179 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-b935a785-8e87-45d0-8915-a7bb457dbc98 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:06.891280 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-b935a785-8e87-45d0-8915-a7bb457dbc98 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:29:06.898772 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-b935a785-8e87-45d0-8915-a7bb457dbc98 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1677 microversion: 2.1 time: 0.152831 Jul 05 10:29:06.899372 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 543/1113] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:29:06 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1677 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:29:07.916041 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8cb2b60f-c05b-47cd-a900-27634cc377e2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:29:07.916662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8cb2b60f-c05b-47cd-a900-27634cc377e2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:29:07.923431 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8cb2b60f-c05b-47cd-a900-27634cc377e2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:07.923886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8cb2b60f-c05b-47cd-a900-27634cc377e2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:07.924677 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8cb2b60f-c05b-47cd-a900-27634cc377e2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:07.925115 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8cb2b60f-c05b-47cd-a900-27634cc377e2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:08.050205 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8cb2b60f-c05b-47cd-a900-27634cc377e2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:29:08.057571 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8cb2b60f-c05b-47cd-a900-27634cc377e2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1677 microversion: 2.1 time: 0.145637 Jul 05 10:29:08.058124 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 571/1114] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:29:07 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1677 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:29:09.075016 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-297dd55a-131f-4b52-b758-c8373c3256ab tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:29:09.075637 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-297dd55a-131f-4b52-b758-c8373c3256ab tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:29:09.082257 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-297dd55a-131f-4b52-b758-c8373c3256ab tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:09.082717 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-297dd55a-131f-4b52-b758-c8373c3256ab tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:09.083486 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-297dd55a-131f-4b52-b758-c8373c3256ab tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:09.083968 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-297dd55a-131f-4b52-b758-c8373c3256ab tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:09.208862 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-297dd55a-131f-4b52-b758-c8373c3256ab tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:29:09.216791 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-297dd55a-131f-4b52-b758-c8373c3256ab tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1654 microversion: 2.1 time: 0.145669 Jul 05 10:29:09.217358 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 544/1115] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:29:09 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1654 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:29:09.233438 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-ba987568-c3b7-4aea-853e-46d7d1b7f861 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:29:09.239865 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ba987568-c3b7-4aea-853e-46d7d1b7f861 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:09.240308 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-ba987568-c3b7-4aea-853e-46d7d1b7f861 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:09.300266 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-ba987568-c3b7-4aea-853e-46d7d1b7f861 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Going to try to start instance {{(pid=117813) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 05 10:29:09.339942 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-ba987568-c3b7-4aea-853e-46d7d1b7f861 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "POST /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/action" status: 202 len: 0 microversion: 2.1 time: 0.110676 Jul 05 10:29:09.340599 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 572/1116] 10.222.0.20 () {60 vars in 1277 bytes} [Tue Jul 5 10:29:09 2022] POST /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/action => generated 0 bytes in 112 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:29:09.349681 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-03a36102-057f-4d99-88f5-386ef9e6201f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:29:09.350387 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-03a36102-057f-4d99-88f5-386ef9e6201f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:29:09.359171 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-03a36102-057f-4d99-88f5-386ef9e6201f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:09.359893 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-03a36102-057f-4d99-88f5-386ef9e6201f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:09.360941 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-03a36102-057f-4d99-88f5-386ef9e6201f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:09.361972 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-03a36102-057f-4d99-88f5-386ef9e6201f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:09.529903 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-03a36102-057f-4d99-88f5-386ef9e6201f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:29:09.538127 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-03a36102-057f-4d99-88f5-386ef9e6201f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1663 microversion: 2.1 time: 0.193189 Jul 05 10:29:09.538703 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 545/1117] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:29:09 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1663 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:29:10.556137 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d9bbda18-53e8-4928-85d7-ab5d73ffee09 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:29:10.556862 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d9bbda18-53e8-4928-85d7-ab5d73ffee09 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:29:10.563559 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d9bbda18-53e8-4928-85d7-ab5d73ffee09 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:10.564004 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d9bbda18-53e8-4928-85d7-ab5d73ffee09 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:10.564793 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d9bbda18-53e8-4928-85d7-ab5d73ffee09 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:29:10.565230 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d9bbda18-53e8-4928-85d7-ab5d73ffee09 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:29:10.693623 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-d9bbda18-53e8-4928-85d7-ab5d73ffee09 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:29:10.701058 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d9bbda18-53e8-4928-85d7-ab5d73ffee09 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1667 microversion: 2.1 time: 0.149750 Jul 05 10:29:10.701614 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 573/1118] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:29:10 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1667 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:14.017609 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=14 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:14.018424 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=46,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:14.022658 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-657447eb-a899-4de4-97e2-05ae2df872a3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:14.023634 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-657447eb-a899-4de4-97e2-05ae2df872a3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:14.031795 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-657447eb-a899-4de4-97e2-05ae2df872a3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:14.032249 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-657447eb-a899-4de4-97e2-05ae2df872a3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:14.033054 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-657447eb-a899-4de4-97e2-05ae2df872a3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:14.033497 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-657447eb-a899-4de4-97e2-05ae2df872a3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:14.064409 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-657447eb-a899-4de4-97e2-05ae2df872a3 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048552 Jul 05 10:30:14.064964 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 546/1119] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:30:14 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 05 10:30:14.119254 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=16 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:14.119993 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=55,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:14.124003 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-67c63800-8043-4669-bd26-ac8f82bc4393 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:14.124536 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-67c63800-8043-4669-bd26-ac8f82bc4393 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:14.131927 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-67c63800-8043-4669-bd26-ac8f82bc4393 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:14.132376 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-67c63800-8043-4669-bd26-ac8f82bc4393 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:14.133160 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-67c63800-8043-4669-bd26-ac8f82bc4393 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:14.133604 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-67c63800-8043-4669-bd26-ac8f82bc4393 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:14.161964 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-67c63800-8043-4669-bd26-ac8f82bc4393 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Going to try to terminate instance {{(pid=117813) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:30:14.234522 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-67c63800-8043-4669-bd26-ac8f82bc4393 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 204 len: 0 microversion: 2.1 time: 0.116647 Jul 05 10:30:14.235021 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 574/1120] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:30:14 2022] DELETE /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:30:14.243624 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-280abaae-96d4-496f-bb0c-485bbdffc312 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:14.244224 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-280abaae-96d4-496f-bb0c-485bbdffc312 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:14.250160 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-280abaae-96d4-496f-bb0c-485bbdffc312 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:14.250605 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-280abaae-96d4-496f-bb0c-485bbdffc312 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:14.251351 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-280abaae-96d4-496f-bb0c-485bbdffc312 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:14.251784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-280abaae-96d4-496f-bb0c-485bbdffc312 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:14.383922 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-280abaae-96d4-496f-bb0c-485bbdffc312 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:14.391760 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-280abaae-96d4-496f-bb0c-485bbdffc312 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1673 microversion: 2.1 time: 0.151959 Jul 05 10:30:14.392323 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 547/1121] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:14 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1673 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:15.409683 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-46680de6-5cae-4573-bc33-d61bb579ec2b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:15.410376 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-46680de6-5cae-4573-bc33-d61bb579ec2b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:15.416934 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-46680de6-5cae-4573-bc33-d61bb579ec2b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:15.417384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-46680de6-5cae-4573-bc33-d61bb579ec2b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:15.418777 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-46680de6-5cae-4573-bc33-d61bb579ec2b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:15.419211 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-46680de6-5cae-4573-bc33-d61bb579ec2b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:15.546174 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-46680de6-5cae-4573-bc33-d61bb579ec2b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:15.553621 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-46680de6-5cae-4573-bc33-d61bb579ec2b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1673 microversion: 2.1 time: 0.148500 Jul 05 10:30:15.554185 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 575/1122] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:15 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1673 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:16.570620 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-21f4d1b5-d937-43bc-8fed-64b4eb751e9f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:16.571178 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-21f4d1b5-d937-43bc-8fed-64b4eb751e9f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:16.577830 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-21f4d1b5-d937-43bc-8fed-64b4eb751e9f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:16.578288 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-21f4d1b5-d937-43bc-8fed-64b4eb751e9f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:16.579051 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-21f4d1b5-d937-43bc-8fed-64b4eb751e9f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:16.579489 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-21f4d1b5-d937-43bc-8fed-64b4eb751e9f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:16.711730 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-21f4d1b5-d937-43bc-8fed-64b4eb751e9f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:16.719286 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-21f4d1b5-d937-43bc-8fed-64b4eb751e9f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1673 microversion: 2.1 time: 0.152506 Jul 05 10:30:16.719865 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 548/1123] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:16 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1673 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:17.527731 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-414194a2-772f-4194-a5cc-537a6b846fc2 req-20676c4f-bda4-4ad1-9384-26c8a6bbed44 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a65a032c-5b68-4e65-b02c-d76e6547c26c", "name": "network-vif-deleted", "tag": "b6c06b3b-2cbc-4544-9d94-78a4a3d5a141"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:30:17.542066 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-414194a2-772f-4194-a5cc-537a6b846fc2 req-20676c4f-bda4-4ad1-9384-26c8a6bbed44 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:17.542615 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-414194a2-772f-4194-a5cc-537a6b846fc2 req-20676c4f-bda4-4ad1-9384-26c8a6bbed44 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:17.558669 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-414194a2-772f-4194-a5cc-537a6b846fc2 req-20676c4f-bda4-4ad1-9384-26c8a6bbed44 service nova] Creating event network-vif-deleted:b6c06b3b-2cbc-4544-9d94-78a4a3d5a141 for instance a65a032c-5b68-4e65-b02c-d76e6547c26c on hv-cinder-maste Jul 05 10:30:17.566374 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-414194a2-772f-4194-a5cc-537a6b846fc2 req-20676c4f-bda4-4ad1-9384-26c8a6bbed44 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042966 Jul 05 10:30:17.566988 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 576/1124] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:30:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:30:17.736875 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-24fb3e69-76ed-4741-af8c-2f24067bd13e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:17.737481 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-24fb3e69-76ed-4741-af8c-2f24067bd13e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:17.743602 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-24fb3e69-76ed-4741-af8c-2f24067bd13e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:17.744049 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-24fb3e69-76ed-4741-af8c-2f24067bd13e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:17.744838 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-24fb3e69-76ed-4741-af8c-2f24067bd13e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:17.745274 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-24fb3e69-76ed-4741-af8c-2f24067bd13e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:17.777434 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-24fb3e69-76ed-4741-af8c-2f24067bd13e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:17.777915 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-24fb3e69-76ed-4741-af8c-2f24067bd13e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:17.778358 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-24fb3e69-76ed-4741-af8c-2f24067bd13e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lazy-loading 'fault' on Instance uuid a65a032c-5b68-4e65-b02c-d76e6547c26c {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:30:17.822380 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-24fb3e69-76ed-4741-af8c-2f24067bd13e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:17.830792 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-24fb3e69-76ed-4741-af8c-2f24067bd13e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 200 len: 1327 microversion: 2.1 time: 0.097543 Jul 05 10:30:17.831254 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 549/1125] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:17 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 1327 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:18.849794 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-4f305108-4cd7-48a0-ba61-7418af0b3484 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:18.850397 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-4f305108-4cd7-48a0-ba61-7418af0b3484 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:18.856542 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f305108-4cd7-48a0-ba61-7418af0b3484 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:18.856987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f305108-4cd7-48a0-ba61-7418af0b3484 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:18.857738 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f305108-4cd7-48a0-ba61-7418af0b3484 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:18.858168 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-4f305108-4cd7-48a0-ba61-7418af0b3484 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:18.870981 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-4f305108-4cd7-48a0-ba61-7418af0b3484 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] HTTP exception thrown: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-4f305108-4cd7-48a0-ba61-7418af0b3484 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Request method failure captured: Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c HTTP/1.1 Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixBGxtukDdXChxNGcBf9KIYDoYTTOhMKKx8ma5aSkC_edsEMwgjZenXD_0fRHl_3y99r5Di-3mSh8hvcSudbvKqdkmjXW-qUepUKEtxJjiRlFxE4KvwEcYupkYfYyEbI7BcaMd5v0JHgQRSF1xD4XX5OzLrDMw1GdWjNZlwRN9Y6Rmsk Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 8b5c262b91db42cb9e4042b180f9fb4d Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:30:18.872577 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:30:18.874357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/8b5c262b91db42cb9e4042b180f9fb4d"}]}] Jul 05 10:30:18.874357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:30:18.874357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 8b5c262b91db42cb9e4042b180f9fb4d Jul 05 10:30:18.874357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:30:18.874357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeTestJSON-1564756241-project Jul 05 10:30:18.874357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:30:18.874357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:30:18.874357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: b131286ad9144575a13972e25cab20db Jul 05 10:30:18.874357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeTestJSON-1564756241-project Jul 05 10:30:18.874357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:30:18.874357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:30:18.874357 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'a65a032c-5b68-4e65-b02c-d76e6547c26c'} Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:30:18.875433 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:30:18.877140 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:18.878579 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-4f305108-4cd7-48a0-ba61-7418af0b3484 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Returning 404 to user: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:30:18.878579 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-4f305108-4cd7-48a0-ba61-7418af0b3484 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 404 len: 111 microversion: 2.1 time: 0.028883 Jul 05 10:30:18.878579 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 577/1126] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:18 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:30:18.882507 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-a8137da5-c274-4080-97d4-d59453a8f645 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '.version_select at 0x7feb894fcb80>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:18.903311 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-a8137da5-c274-4080-97d4-d59453a8f645 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1739685357" status: 202 len: 0 microversion: 2.1 time: 0.024513 Jul 05 10:30:18.903883 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 550/1127] 10.222.0.20 () {58 vars in 1220 bytes} [Tue Jul 5 10:30:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1739685357 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:30:19.499837 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2b7c3f36-f885-4eae-8053-1388f73e8d63 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Action: 'create', calling method: .version_select at 0x7feb8df38430>, body: {"keypair": {"name": "tempest-keypair-1316808162", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK9VDmYd9SHxQqZjlbYrjef/mONDEPiohH+RU2NtWRvCSdSfdat3hpW+ScV1dLm+RwhYuFy5NUrO86g7VgEiGWCqyz6M0SfbzNOzkdKfakI3yNqC708VN36rsDv7MvfygQ=="}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:30:19.514192 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-2b7c3f36-f885-4eae-8053-1388f73e8d63 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Getting quotas for project 8b5c262b91db42cb9e4042b180f9fb4d. Resources: {'key_pairs'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:30:19.517950 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.quota [None req-2b7c3f36-f885-4eae-8053-1388f73e8d63 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Getting quotas for user b131286ad9144575a13972e25cab20db and project 8b5c262b91db42cb9e4042b180f9fb4d. Resources: {'key_pairs'} {{(pid=117813) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:30:19.543230 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2b7c3f36-f885-4eae-8053-1388f73e8d63 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.047248 Jul 05 10:30:19.543854 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 578/1128] 10.222.0.20 () {60 vars in 1158 bytes} [Tue Jul 5 10:30:19 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:30:20.721980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-640716276", "imageRef": "a37be4cf-09f2-4eeb-9091-01ea6df6603d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "905e65c9-d2c9-40b0-a3db-38b3be82f3f2"}], "security_groups": [{"name": "tempest-securitygroup--972640809"}], "key_name": "tempest-keypair-1316808162", "user_data": "}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:30:20.823447 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.network.neutron [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] validate_networks() for [('905e65c9-d2c9-40b0-a3db-38b3be82f3f2', None, None, None, None, None)] {{(pid=117812) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 05 10:30:21.190744 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Flavor limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 05 10:30:21.191163 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Image limits 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 05 10:30:21.191630 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Flavor pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 05 10:30:21.192068 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Image pref 0:0:0 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 05 10:30:21.192515 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.virt.hardware [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117812) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 05 10:30:21.201686 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:21.202132 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:21.202925 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:21.203355 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:21.220000 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Getting quotas for project 8b5c262b91db42cb9e4042b180f9fb4d. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 05 10:30:21.223867 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.quota [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Getting quotas for user b131286ad9144575a13972e25cab20db and project 8b5c262b91db42cb9e4042b180f9fb4d. Resources: {'cores', 'ram', 'instances'} {{(pid=117812) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 05 10:30:21.234618 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Going to run 1 instances... {{(pid=117812) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 05 10:30:21.236841 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a37be4cf-09f2-4eeb-9091-01ea6df6603d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117812) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 05 10:30:21.283325 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3f925ade-19bc-4f35-b0ce-e5004c046873 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.566126 Jul 05 10:30:21.284222 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 551/1129] 10.222.0.20 () {60 vars in 1146 bytes} [Tue Jul 5 10:30:20 2022] POST /compute/v2.1/servers => generated 401 bytes in 568 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 05 10:30:21.294679 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-49720123-06a3-4ea5-a72a-a2bee65f9cc9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:21.295389 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-49720123-06a3-4ea5-a72a-a2bee65f9cc9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:21.352420 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-49720123-06a3-4ea5-a72a-a2bee65f9cc9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:21.359356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-49720123-06a3-4ea5-a72a-a2bee65f9cc9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1237 microversion: 2.1 time: 0.069084 Jul 05 10:30:21.360232 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 579/1130] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:21 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1237 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:22.376968 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-4b94f776-59dc-4dee-a6c3-e0e26d332cac tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:22.377589 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-4b94f776-59dc-4dee-a6c3-e0e26d332cac tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:22.383690 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4b94f776-59dc-4dee-a6c3-e0e26d332cac tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:22.384149 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4b94f776-59dc-4dee-a6c3-e0e26d332cac tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:22.384960 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4b94f776-59dc-4dee-a6c3-e0e26d332cac tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:22.386012 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-4b94f776-59dc-4dee-a6c3-e0e26d332cac tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:22.451228 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-4b94f776-59dc-4dee-a6c3-e0e26d332cac tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:22.458727 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-4b94f776-59dc-4dee-a6c3-e0e26d332cac tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1295 microversion: 2.1 time: 0.086107 Jul 05 10:30:22.459287 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 552/1131] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:22 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1295 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:23.480292 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-405018a5-402e-4f85-9a14-2e86e4cdb008 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:23.481621 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-405018a5-402e-4f85-9a14-2e86e4cdb008 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:23.488504 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-405018a5-402e-4f85-9a14-2e86e4cdb008 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:23.489042 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-405018a5-402e-4f85-9a14-2e86e4cdb008 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:23.490469 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-405018a5-402e-4f85-9a14-2e86e4cdb008 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:23.490984 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-405018a5-402e-4f85-9a14-2e86e4cdb008 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:23.709750 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-405018a5-402e-4f85-9a14-2e86e4cdb008 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:23.717969 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-405018a5-402e-4f85-9a14-2e86e4cdb008 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1362 microversion: 2.1 time: 0.245809 Jul 05 10:30:23.719906 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 580/1132] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:23 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1362 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:23.781212 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-d24b3207-5874-4986-b6d9-bdd933a46a5d req-ee39c795-3f06-40ef-a494-fea4789dd6e3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bc953afb-2d84-4da4-9478-21f39ca83773", "tag": "8ccb0cb8-059d-462f-8560-f28cdf748acd"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:30:23.793231 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-d24b3207-5874-4986-b6d9-bdd933a46a5d req-ee39c795-3f06-40ef-a494-fea4789dd6e3 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:23.794132 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-d24b3207-5874-4986-b6d9-bdd933a46a5d req-ee39c795-3f06-40ef-a494-fea4789dd6e3 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:23.808414 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-d24b3207-5874-4986-b6d9-bdd933a46a5d req-ee39c795-3f06-40ef-a494-fea4789dd6e3 service nova] Creating event network-changed:8ccb0cb8-059d-462f-8560-f28cdf748acd for instance bc953afb-2d84-4da4-9478-21f39ca83773 on hv-cinder-maste Jul 05 10:30:23.815070 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-d24b3207-5874-4986-b6d9-bdd933a46a5d req-ee39c795-3f06-40ef-a494-fea4789dd6e3 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039536 Jul 05 10:30:23.815677 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 553/1133] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:30:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:30:24.736485 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-08e9afc1-55c8-4764-93b3-1de374e20884 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:24.737105 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-08e9afc1-55c8-4764-93b3-1de374e20884 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:24.743693 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-08e9afc1-55c8-4764-93b3-1de374e20884 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:24.744143 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-08e9afc1-55c8-4764-93b3-1de374e20884 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:24.744959 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-08e9afc1-55c8-4764-93b3-1de374e20884 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:24.745904 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-08e9afc1-55c8-4764-93b3-1de374e20884 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:24.872444 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-08e9afc1-55c8-4764-93b3-1de374e20884 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:24.880001 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-08e9afc1-55c8-4764-93b3-1de374e20884 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1362 microversion: 2.1 time: 0.147459 Jul 05 10:30:24.880591 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 581/1134] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:24 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1362 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:25.897486 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5d3b13ab-8a2f-4901-a911-0cb07e7e273e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:25.898190 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5d3b13ab-8a2f-4901-a911-0cb07e7e273e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:25.904987 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5d3b13ab-8a2f-4901-a911-0cb07e7e273e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:25.905444 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5d3b13ab-8a2f-4901-a911-0cb07e7e273e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:25.906213 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5d3b13ab-8a2f-4901-a911-0cb07e7e273e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:25.906662 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5d3b13ab-8a2f-4901-a911-0cb07e7e273e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:26.034266 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5d3b13ab-8a2f-4901-a911-0cb07e7e273e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:26.041698 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5d3b13ab-8a2f-4901-a911-0cb07e7e273e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1362 microversion: 2.1 time: 0.148401 Jul 05 10:30:26.042261 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 554/1135] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:25 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1362 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:27.059473 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-f2a255b8-a8e8-407e-8a3c-43df4700a58a tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:27.060085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-f2a255b8-a8e8-407e-8a3c-43df4700a58a tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:27.066428 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f2a255b8-a8e8-407e-8a3c-43df4700a58a tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:27.066858 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f2a255b8-a8e8-407e-8a3c-43df4700a58a tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:27.067625 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f2a255b8-a8e8-407e-8a3c-43df4700a58a tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:27.068057 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-f2a255b8-a8e8-407e-8a3c-43df4700a58a tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:27.319085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-f2a255b8-a8e8-407e-8a3c-43df4700a58a tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:27.328313 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-f2a255b8-a8e8-407e-8a3c-43df4700a58a tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1362 microversion: 2.1 time: 0.272949 Jul 05 10:30:27.329529 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 582/1136] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:27 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1362 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:27.961349 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-aaeee661-80ee-4d91-a77c-768d0d16c391 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:27.966557 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-d43f483f-5be1-4957-a55e-1956454b478f req-33c543ce-49fe-4694-abea-726b952f946e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e91854f0-736f-436d-ad34-11e66f585c5d", "tag": "dc432f06-5b7d-409d-a315-06582b118d3d"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:30:27.970951 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-aaeee661-80ee-4d91-a77c-768d0d16c391 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:27.974771 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-d43f483f-5be1-4957-a55e-1956454b478f req-33c543ce-49fe-4694-abea-726b952f946e service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:27.975212 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-d43f483f-5be1-4957-a55e-1956454b478f req-33c543ce-49fe-4694-abea-726b952f946e service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:27.977423 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-aaeee661-80ee-4d91-a77c-768d0d16c391 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:27.978048 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-aaeee661-80ee-4d91-a77c-768d0d16c391 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:27.979003 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-aaeee661-80ee-4d91-a77c-768d0d16c391 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:27.979441 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-aaeee661-80ee-4d91-a77c-768d0d16c391 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:27.986745 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-d43f483f-5be1-4957-a55e-1956454b478f req-33c543ce-49fe-4694-abea-726b952f946e service nova] Creating event network-changed:dc432f06-5b7d-409d-a315-06582b118d3d for instance e91854f0-736f-436d-ad34-11e66f585c5d on hv-cinder-maste Jul 05 10:30:27.993513 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-d43f483f-5be1-4957-a55e-1956454b478f req-33c543ce-49fe-4694-abea-726b952f946e service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030414 Jul 05 10:30:27.994297 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 583/1137] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:30:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:30:28.007695 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-aaeee661-80ee-4d91-a77c-768d0d16c391 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Going to try to terminate instance {{(pid=117812) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:30:28.088835 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-aaeee661-80ee-4d91-a77c-768d0d16c391 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "DELETE /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 204 len: 0 microversion: 2.1 time: 0.130900 Jul 05 10:30:28.089497 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 555/1138] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:30:27 2022] DELETE /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:30:28.101151 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-7e2ac563-1b9c-4592-83b1-896311d1178e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:28.102663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7e2ac563-1b9c-4592-83b1-896311d1178e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:28.110247 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7e2ac563-1b9c-4592-83b1-896311d1178e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:28.111127 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7e2ac563-1b9c-4592-83b1-896311d1178e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:28.112245 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7e2ac563-1b9c-4592-83b1-896311d1178e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:28.112897 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7e2ac563-1b9c-4592-83b1-896311d1178e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:28.285021 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-7e2ac563-1b9c-4592-83b1-896311d1178e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:28.293139 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-7e2ac563-1b9c-4592-83b1-896311d1178e tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1576 microversion: 2.1 time: 0.197347 Jul 05 10:30:28.293729 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 584/1139] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:28 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1576 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:28.346661 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-59c7ab86-09e7-41d4-be65-d7d686b8226f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:28.347276 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-59c7ab86-09e7-41d4-be65-d7d686b8226f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:28.356486 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59c7ab86-09e7-41d4-be65-d7d686b8226f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:28.356983 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59c7ab86-09e7-41d4-be65-d7d686b8226f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:28.358287 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59c7ab86-09e7-41d4-be65-d7d686b8226f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:28.358835 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-59c7ab86-09e7-41d4-be65-d7d686b8226f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:28.503184 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-59c7ab86-09e7-41d4-be65-d7d686b8226f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:28.510854 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-59c7ab86-09e7-41d4-be65-d7d686b8226f tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1362 microversion: 2.1 time: 0.168486 Jul 05 10:30:28.511426 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 556/1140] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:28 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1362 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:29.321292 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8068c8a9-c3ae-4eda-8683-ce752da25c77 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:29.322662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8068c8a9-c3ae-4eda-8683-ce752da25c77 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:29.335358 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8068c8a9-c3ae-4eda-8683-ce752da25c77 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:29.336335 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8068c8a9-c3ae-4eda-8683-ce752da25c77 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:29.337345 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8068c8a9-c3ae-4eda-8683-ce752da25c77 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:29.338493 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8068c8a9-c3ae-4eda-8683-ce752da25c77 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:29.527794 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-5f8a0751-a7f4-4c65-bdcb-db6e6837b29e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:29.528473 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-5f8a0751-a7f4-4c65-bdcb-db6e6837b29e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:29.535317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5f8a0751-a7f4-4c65-bdcb-db6e6837b29e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:29.535761 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5f8a0751-a7f4-4c65-bdcb-db6e6837b29e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:29.536743 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5f8a0751-a7f4-4c65-bdcb-db6e6837b29e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:29.537350 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-5f8a0751-a7f4-4c65-bdcb-db6e6837b29e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:29.561912 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-8068c8a9-c3ae-4eda-8683-ce752da25c77 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:29.569875 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8068c8a9-c3ae-4eda-8683-ce752da25c77 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1576 microversion: 2.1 time: 0.259872 Jul 05 10:30:29.570727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 585/1141] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:29 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1576 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:29.669090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-5f8a0751-a7f4-4c65-bdcb-db6e6837b29e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:29.677429 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-5f8a0751-a7f4-4c65-bdcb-db6e6837b29e tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1362 microversion: 2.1 time: 0.154124 Jul 05 10:30:29.678007 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 557/1142] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:29 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1362 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:30.388034 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-36d39dad-5a54-43e9-945b-84eebb3923b2 req-5dc7856c-ee7c-4a82-859a-d5d1719b8b64 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e91854f0-736f-436d-ad34-11e66f585c5d", "name": "network-vif-deleted", "tag": "dc432f06-5b7d-409d-a315-06582b118d3d"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:30:30.396962 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-36d39dad-5a54-43e9-945b-84eebb3923b2 req-5dc7856c-ee7c-4a82-859a-d5d1719b8b64 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:30.397402 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-36d39dad-5a54-43e9-945b-84eebb3923b2 req-5dc7856c-ee7c-4a82-859a-d5d1719b8b64 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:30.409429 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-36d39dad-5a54-43e9-945b-84eebb3923b2 req-5dc7856c-ee7c-4a82-859a-d5d1719b8b64 service nova] Creating event network-vif-deleted:dc432f06-5b7d-409d-a315-06582b118d3d for instance e91854f0-736f-436d-ad34-11e66f585c5d on hv-cinder-maste Jul 05 10:30:30.414971 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-36d39dad-5a54-43e9-945b-84eebb3923b2 req-5dc7856c-ee7c-4a82-859a-d5d1719b8b64 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030326 Jul 05 10:30:30.415772 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 586/1143] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:30:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:30:30.442580 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-276750d8-387c-4abf-9827-7160e87c7d44 req-8c404b82-ed28-442b-96ac-d1c5d3a301bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bc953afb-2d84-4da4-9478-21f39ca83773", "name": "network-vif-plugged", "status": "completed", "tag": "8ccb0cb8-059d-462f-8560-f28cdf748acd"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:30:30.453377 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-276750d8-387c-4abf-9827-7160e87c7d44 req-8c404b82-ed28-442b-96ac-d1c5d3a301bc service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:30.453819 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-276750d8-387c-4abf-9827-7160e87c7d44 req-8c404b82-ed28-442b-96ac-d1c5d3a301bc service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:30.467188 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-276750d8-387c-4abf-9827-7160e87c7d44 req-8c404b82-ed28-442b-96ac-d1c5d3a301bc service nova] Creating event network-vif-plugged:8ccb0cb8-059d-462f-8560-f28cdf748acd for instance bc953afb-2d84-4da4-9478-21f39ca83773 on hv-cinder-maste Jul 05 10:30:30.473180 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-276750d8-387c-4abf-9827-7160e87c7d44 req-8c404b82-ed28-442b-96ac-d1c5d3a301bc service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034674 Jul 05 10:30:30.474903 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 558/1144] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:30:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:30:30.587653 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-b94b72eb-43c6-49e8-a242-07667a77e0d9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:30.588483 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-b94b72eb-43c6-49e8-a242-07667a77e0d9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:30.595285 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b94b72eb-43c6-49e8-a242-07667a77e0d9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:30.595805 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b94b72eb-43c6-49e8-a242-07667a77e0d9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:30.596760 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b94b72eb-43c6-49e8-a242-07667a77e0d9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:30.597181 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-b94b72eb-43c6-49e8-a242-07667a77e0d9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:30.673740 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-b94b72eb-43c6-49e8-a242-07667a77e0d9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:30.686704 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-b94b72eb-43c6-49e8-a242-07667a77e0d9 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1225 microversion: 2.1 time: 0.103142 Jul 05 10:30:30.687570 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 587/1145] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:30 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:30.694635 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-22f2931e-ee18-4736-aa34-62b234a9b967 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:30.695329 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-22f2931e-ee18-4736-aa34-62b234a9b967 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:30.702861 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22f2931e-ee18-4736-aa34-62b234a9b967 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:30.703317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22f2931e-ee18-4736-aa34-62b234a9b967 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:30.704096 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22f2931e-ee18-4736-aa34-62b234a9b967 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:30.704559 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-22f2931e-ee18-4736-aa34-62b234a9b967 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:30.857615 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-22f2931e-ee18-4736-aa34-62b234a9b967 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:30.865024 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-22f2931e-ee18-4736-aa34-62b234a9b967 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1362 microversion: 2.1 time: 0.174531 Jul 05 10:30:30.865587 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 559/1146] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:30 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1362 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:31.703588 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-7107d8cd-85a2-4572-baf3-bd0bf410f432 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:31.704159 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7107d8cd-85a2-4572-baf3-bd0bf410f432 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:31.705995 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell0:SELECT=6 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:31.712041 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7107d8cd-85a2-4572-baf3-bd0bf410f432 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:31.712553 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7107d8cd-85a2-4572-baf3-bd0bf410f432 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:31.713321 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7107d8cd-85a2-4572-baf3-bd0bf410f432 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:31.713759 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7107d8cd-85a2-4572-baf3-bd0bf410f432 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:31.744138 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7107d8cd-85a2-4572-baf3-bd0bf410f432 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:31.744632 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7107d8cd-85a2-4572-baf3-bd0bf410f432 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:31.745078 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.objects.instance [None req-7107d8cd-85a2-4572-baf3-bd0bf410f432 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lazy-loading 'fault' on Instance uuid e91854f0-736f-436d-ad34-11e66f585c5d {{(pid=117813) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:30:31.788589 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-7107d8cd-85a2-4572-baf3-bd0bf410f432 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:31.796657 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-7107d8cd-85a2-4572-baf3-bd0bf410f432 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 200 len: 1230 microversion: 2.1 time: 0.097098 Jul 05 10:30:31.797227 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 588/1147] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:31 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 1230 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:31.881800 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-e18133f4-f34f-4ce6-a667-bee700c3b7ca tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:31.882379 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-e18133f4-f34f-4ce6-a667-bee700c3b7ca tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:31.884346 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=2 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:31.885035 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell0:SELECT=6 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:31.891724 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e18133f4-f34f-4ce6-a667-bee700c3b7ca tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:31.892182 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e18133f4-f34f-4ce6-a667-bee700c3b7ca tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:31.892985 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e18133f4-f34f-4ce6-a667-bee700c3b7ca tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:31.893433 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-e18133f4-f34f-4ce6-a667-bee700c3b7ca tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:32.022747 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-e18133f4-f34f-4ce6-a667-bee700c3b7ca tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:32.031382 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-e18133f4-f34f-4ce6-a667-bee700c3b7ca tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1362 microversion: 2.1 time: 0.153342 Jul 05 10:30:32.037493 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 560/1148] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:31 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1362 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:32.813812 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1d680fa3-785d-49a6-8934-be3dd9be0606 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:32.814409 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-1d680fa3-785d-49a6-8934-be3dd9be0606 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: e91854f0-736f-436d-ad34-11e66f585c5d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:32.820871 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1d680fa3-785d-49a6-8934-be3dd9be0606 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:32.821317 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1d680fa3-785d-49a6-8934-be3dd9be0606 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:32.822089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1d680fa3-785d-49a6-8934-be3dd9be0606 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:32.822560 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-1d680fa3-785d-49a6-8934-be3dd9be0606 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:32.836458 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-1d680fa3-785d-49a6-8934-be3dd9be0606 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] HTTP exception thrown: Instance e91854f0-736f-436d-ad34-11e66f585c5d could not be found. Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1d680fa3-785d-49a6-8934-be3dd9be0606 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Request method failure captured: Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d HTTP/1.1 Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixBDpuHveZ3XeN9U67EK4lKdYSp_m_64mU-LBfdZJwGXE8VS4y9KkhlpV-sF73TZRPltmWaOCmj1JJgwzGBK1g-a6vXND_jKDWiyWKsXT9eD5CtGrsU53oavQhY5y4SU9PBsCzPb3si94ZYUpzEQAVskEwEVkAMCsD3v3D2ZU1cRNTYA Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 86eac45249ad4a5e9513fcc091bcf42e Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-TestVolumeBootPattern-620052980 Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:30:32.838100 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:30:32.839789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/86eac45249ad4a5e9513fcc091bcf42e"}]}] Jul 05 10:30:32.839789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-TestVolumeBootPattern-620052980 Jul 05 10:30:32.839789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 86eac45249ad4a5e9513fcc091bcf42e Jul 05 10:30:32.839789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-TestVolumeBootPattern-620052980 Jul 05 10:30:32.839789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-TestVolumeBootPattern-620052980-project Jul 05 10:30:32.839789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:30:32.839789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:30:32.839789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: c9203890ad7649f897b13ac7bdf6be3b Jul 05 10:30:32.839789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-TestVolumeBootPattern-620052980-project Jul 05 10:30:32.839789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:30:32.839789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance e91854f0-736f-436d-ad34-11e66f585c5d could not be found. Jul 05 10:30:32.839789 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'e91854f0-736f-436d-ad34-11e66f585c5d'} Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e91854f0-736f-436d-ad34-11e66f585c5d could not be found. Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:30:32.840809 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e91854f0-736f-436d-ad34-11e66f585c5d could not be found. Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e91854f0-736f-436d-ad34-11e66f585c5d could not be found. Jul 05 10:30:32.842386 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:32.843824 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-1d680fa3-785d-49a6-8934-be3dd9be0606 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Returning 404 to user: Instance e91854f0-736f-436d-ad34-11e66f585c5d could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:30:32.843824 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-1d680fa3-785d-49a6-8934-be3dd9be0606 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d" status: 404 len: 111 microversion: 2.1 time: 0.029681 Jul 05 10:30:32.843824 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 589/1149] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:32 2022] GET /compute/v2.1/servers/e91854f0-736f-436d-ad34-11e66f585c5d => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:30:33.049138 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-361572f3-354b-454d-8e3b-b2f6da80e3f2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:33.049744 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-361572f3-354b-454d-8e3b-b2f6da80e3f2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:33.056161 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-361572f3-354b-454d-8e3b-b2f6da80e3f2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:33.056650 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-361572f3-354b-454d-8e3b-b2f6da80e3f2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:33.057415 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-361572f3-354b-454d-8e3b-b2f6da80e3f2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:33.057854 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-361572f3-354b-454d-8e3b-b2f6da80e3f2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:33.192997 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-361572f3-354b-454d-8e3b-b2f6da80e3f2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:33.200336 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-361572f3-354b-454d-8e3b-b2f6da80e3f2 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1548 microversion: 2.1 time: 0.155052 Jul 05 10:30:33.200943 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 561/1150] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:33 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1548 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:33.216680 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-542087a4-f2f3-4838-bf44-e099fa7e9a0b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:33.217166 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-542087a4-f2f3-4838-bf44-e099fa7e9a0b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:33.223067 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-542087a4-f2f3-4838-bf44-e099fa7e9a0b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:33.223507 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-542087a4-f2f3-4838-bf44-e099fa7e9a0b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:33.224257 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-542087a4-f2f3-4838-bf44-e099fa7e9a0b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:33.224709 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-542087a4-f2f3-4838-bf44-e099fa7e9a0b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:33.306480 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-542087a4-f2f3-4838-bf44-e099fa7e9a0b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.093574 Jul 05 10:30:33.307072 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 590/1151] 10.222.0.20 () {58 vars in 1274 bytes} [Tue Jul 5 10:30:33 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-interface => generated 284 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:30:34.473406 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-f4b3a3c8-b20a-47f5-b23f-0162ba7a9ffa req-37cc9cc8-4726-4271-905b-0c42db1ed660 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bc953afb-2d84-4da4-9478-21f39ca83773", "tag": "8ccb0cb8-059d-462f-8560-f28cdf748acd"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:30:34.482418 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-f4b3a3c8-b20a-47f5-b23f-0162ba7a9ffa req-37cc9cc8-4726-4271-905b-0c42db1ed660 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:34.482868 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-f4b3a3c8-b20a-47f5-b23f-0162ba7a9ffa req-37cc9cc8-4726-4271-905b-0c42db1ed660 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:34.493654 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-f4b3a3c8-b20a-47f5-b23f-0162ba7a9ffa req-37cc9cc8-4726-4271-905b-0c42db1ed660 service nova] Creating event network-changed:8ccb0cb8-059d-462f-8560-f28cdf748acd for instance bc953afb-2d84-4da4-9478-21f39ca83773 on hv-cinder-maste Jul 05 10:30:34.499321 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-f4b3a3c8-b20a-47f5-b23f-0162ba7a9ffa req-37cc9cc8-4726-4271-905b-0c42db1ed660 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029948 Jul 05 10:30:34.499914 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 562/1152] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:30:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:30:35.908806 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-240feab5-3675-4bee-810f-9ba549bfc94a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:35.909296 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-240feab5-3675-4bee-810f-9ba549bfc94a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:35.915947 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-240feab5-3675-4bee-810f-9ba549bfc94a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:35.916416 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-240feab5-3675-4bee-810f-9ba549bfc94a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:35.917244 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-240feab5-3675-4bee-810f-9ba549bfc94a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:35.917687 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-240feab5-3675-4bee-810f-9ba549bfc94a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:35.944446 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-240feab5-3675-4bee-810f-9ba549bfc94a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Going to try to terminate instance {{(pid=117813) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:30:36.024149 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-240feab5-3675-4bee-810f-9ba549bfc94a tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "DELETE /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 204 len: 0 microversion: 2.1 time: 0.119023 Jul 05 10:30:36.024865 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 591/1153] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:30:35 2022] DELETE /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:30:36.034115 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-45e0c65c-dece-43b1-9f13-87ac5134fcfc tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:36.034933 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-45e0c65c-dece-43b1-9f13-87ac5134fcfc tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:36.041474 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-45e0c65c-dece-43b1-9f13-87ac5134fcfc tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:36.042021 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-45e0c65c-dece-43b1-9f13-87ac5134fcfc tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:36.043353 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-45e0c65c-dece-43b1-9f13-87ac5134fcfc tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:36.044000 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-45e0c65c-dece-43b1-9f13-87ac5134fcfc tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:36.178770 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-45e0c65c-dece-43b1-9f13-87ac5134fcfc tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:36.186396 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-45e0c65c-dece-43b1-9f13-87ac5134fcfc tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1578 microversion: 2.1 time: 0.156876 Jul 05 10:30:36.186980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 563/1154] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:36 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1578 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:36.514974 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-c3fc0c3e-972f-476b-b118-8df7fcbffc3a req-cd172dfc-c639-4934-8395-b8f7071f31a9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b87058af-a15a-4a2b-9f05-e3f9b99e4a4d", "tag": "bc97aaf3-0c0d-415c-b3c3-645269085c58"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:30:36.522987 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-c3fc0c3e-972f-476b-b118-8df7fcbffc3a req-cd172dfc-c639-4934-8395-b8f7071f31a9 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:36.523428 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-c3fc0c3e-972f-476b-b118-8df7fcbffc3a req-cd172dfc-c639-4934-8395-b8f7071f31a9 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:36.534870 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-c3fc0c3e-972f-476b-b118-8df7fcbffc3a req-cd172dfc-c639-4934-8395-b8f7071f31a9 service nova] Creating event network-changed:bc97aaf3-0c0d-415c-b3c3-645269085c58 for instance b87058af-a15a-4a2b-9f05-e3f9b99e4a4d on hv-cinder-maste Jul 05 10:30:36.542236 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-c3fc0c3e-972f-476b-b118-8df7fcbffc3a req-cd172dfc-c639-4934-8395-b8f7071f31a9 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031997 Jul 05 10:30:36.542860 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 592/1155] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:30:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:30:37.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-fd7a525c-ac92-4eac-a03a-fba401a5c788 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:37.204615 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-fd7a525c-ac92-4eac-a03a-fba401a5c788 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:37.211018 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fd7a525c-ac92-4eac-a03a-fba401a5c788 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:37.211469 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fd7a525c-ac92-4eac-a03a-fba401a5c788 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:37.212229 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fd7a525c-ac92-4eac-a03a-fba401a5c788 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:37.212705 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-fd7a525c-ac92-4eac-a03a-fba401a5c788 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:37.217356 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:37.426010 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-fd7a525c-ac92-4eac-a03a-fba401a5c788 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:37.433694 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-fd7a525c-ac92-4eac-a03a-fba401a5c788 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1578 microversion: 2.1 time: 0.234308 Jul 05 10:30:37.434253 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 564/1156] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:37 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1578 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:38.451962 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-3d9da15e-ca9c-4ab2-be5b-e779840d6d75 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:38.452603 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-3d9da15e-ca9c-4ab2-be5b-e779840d6d75 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:38.459723 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3d9da15e-ca9c-4ab2-be5b-e779840d6d75 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:38.460168 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3d9da15e-ca9c-4ab2-be5b-e779840d6d75 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:38.460974 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3d9da15e-ca9c-4ab2-be5b-e779840d6d75 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:38.461381 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-3d9da15e-ca9c-4ab2-be5b-e779840d6d75 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:38.465924 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:30:38.564774 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [req-c3662364-42b5-4d83-86d0-ade55a441138 req-4e5358fd-a272-4cd1-97a9-1c5319beaf87 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b87058af-a15a-4a2b-9f05-e3f9b99e4a4d", "name": "network-vif-deleted", "tag": "bc97aaf3-0c0d-415c-b3c3-645269085c58"}]} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:30:38.573989 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-c3662364-42b5-4d83-86d0-ade55a441138 req-4e5358fd-a272-4cd1-97a9-1c5319beaf87 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:38.574546 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [req-c3662364-42b5-4d83-86d0-ade55a441138 req-4e5358fd-a272-4cd1-97a9-1c5319beaf87 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:38.580061 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-3d9da15e-ca9c-4ab2-be5b-e779840d6d75 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c9203890ad7649f897b13ac7bdf6be3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86eac45249ad4a5e9513fcc091bcf42e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:30:38.585679 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.compute.server_external_events [req-c3662364-42b5-4d83-86d0-ade55a441138 req-4e5358fd-a272-4cd1-97a9-1c5319beaf87 service nova] Creating event network-vif-deleted:bc97aaf3-0c0d-415c-b3c3-645269085c58 for instance b87058af-a15a-4a2b-9f05-e3f9b99e4a4d on hv-cinder-maste Jul 05 10:30:38.587679 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-3d9da15e-ca9c-4ab2-be5b-e779840d6d75 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 200 len: 1226 microversion: 2.1 time: 0.140534 Jul 05 10:30:38.588239 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 593/1157] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:38 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 1226 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:30:38.590924 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [req-c3662364-42b5-4d83-86d0-ade55a441138 req-4e5358fd-a272-4cd1-97a9-1c5319beaf87 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030499 Jul 05 10:30:38.591506 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 565/1158] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:30:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 05 10:30:39.604230 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2b231bf6-7010-48d5-abe9-9d9e326a7841 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:39.604846 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-2b231bf6-7010-48d5-abe9-9d9e326a7841 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: b87058af-a15a-4a2b-9f05-e3f9b99e4a4d] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:39.611308 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2b231bf6-7010-48d5-abe9-9d9e326a7841 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:39.611749 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2b231bf6-7010-48d5-abe9-9d9e326a7841 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:39.612529 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2b231bf6-7010-48d5-abe9-9d9e326a7841 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:39.612963 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-2b231bf6-7010-48d5-abe9-9d9e326a7841 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:39.626775 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-2b231bf6-7010-48d5-abe9-9d9e326a7841 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] HTTP exception thrown: Instance b87058af-a15a-4a2b-9f05-e3f9b99e4a4d could not be found. Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2b231bf6-7010-48d5-abe9-9d9e326a7841 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Request method failure captured: Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d HTTP/1.1 Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixBDpuHveZ3XeN9U67EK4lKdYSp_m_64mU-LBfdZJwGXE8VS4y9KkhlpV-sF73TZRPltmWaOCmj1JJgwzGBK1g-a6vXND_jKDWiyWKsXT9eD5CtGrsU53oavQhY5y4SU9PBsCzPb3si94ZYUpzEQAVskEwEVkAMCsD3v3D2ZU1cRNTYA Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 86eac45249ad4a5e9513fcc091bcf42e Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-TestVolumeBootPattern-620052980 Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:30:39.628356 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:30:39.629886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/86eac45249ad4a5e9513fcc091bcf42e"}]}] Jul 05 10:30:39.629886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-TestVolumeBootPattern-620052980 Jul 05 10:30:39.629886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 86eac45249ad4a5e9513fcc091bcf42e Jul 05 10:30:39.629886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-TestVolumeBootPattern-620052980 Jul 05 10:30:39.629886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-TestVolumeBootPattern-620052980-project Jul 05 10:30:39.629886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:30:39.629886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:30:39.629886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: c9203890ad7649f897b13ac7bdf6be3b Jul 05 10:30:39.629886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-TestVolumeBootPattern-620052980-project Jul 05 10:30:39.629886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:30:39.629886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance b87058af-a15a-4a2b-9f05-e3f9b99e4a4d could not be found. Jul 05 10:30:39.629886 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'b87058af-a15a-4a2b-9f05-e3f9b99e4a4d'} Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b87058af-a15a-4a2b-9f05-e3f9b99e4a4d could not be found. Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:30:39.630727 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b87058af-a15a-4a2b-9f05-e3f9b99e4a4d could not be found. Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b87058af-a15a-4a2b-9f05-e3f9b99e4a4d could not be found. Jul 05 10:30:39.632368 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:39.634052 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-2b231bf6-7010-48d5-abe9-9d9e326a7841 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Returning 404 to user: Instance b87058af-a15a-4a2b-9f05-e3f9b99e4a4d could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:30:39.634052 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-2b231bf6-7010-48d5-abe9-9d9e326a7841 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d" status: 404 len: 111 microversion: 2.1 time: 0.029461 Jul 05 10:30:39.634052 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 594/1159] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:39 2022] GET /compute/v2.1/servers/b87058af-a15a-4a2b-9f05-e3f9b99e4a4d => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:30:40.063308 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0f1f2d5c-d647-40fc-9075-552751be1473 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:40.063883 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0f1f2d5c-d647-40fc-9075-552751be1473 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:40.070051 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0f1f2d5c-d647-40fc-9075-552751be1473 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:40.070497 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0f1f2d5c-d647-40fc-9075-552751be1473 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:40.071251 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0f1f2d5c-d647-40fc-9075-552751be1473 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:40.071686 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0f1f2d5c-d647-40fc-9075-552751be1473 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:40.085069 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-0f1f2d5c-d647-40fc-9075-552751be1473 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] HTTP exception thrown: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0f1f2d5c-d647-40fc-9075-552751be1473 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Request method failure captured: Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: DELETE /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f HTTP/1.1 Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixBDpuHveZ3XeN9U67EK4lKdYSp_m_64mU-LBfdZJwGXE8VS4y9KkhlpV-sF73TZRPltmWaOCmj1JJgwzGBK1g-a6vXND_jKDWiyWKsXT9eD5CtGrsU53oavQhY5y4SU9PBsCzPb3si94ZYUpzEQAVskEwEVkAMCsD3v3D2ZU1cRNTYA Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: 86eac45249ad4a5e9513fcc091bcf42e Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-TestVolumeBootPattern-620052980 Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:30:40.086666 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:30:40.088330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/86eac45249ad4a5e9513fcc091bcf42e"}]}] Jul 05 10:30:40.088330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-TestVolumeBootPattern-620052980 Jul 05 10:30:40.088330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: 86eac45249ad4a5e9513fcc091bcf42e Jul 05 10:30:40.088330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-TestVolumeBootPattern-620052980 Jul 05 10:30:40.088330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-TestVolumeBootPattern-620052980-project Jul 05 10:30:40.088330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:30:40.088330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:30:40.088330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: c9203890ad7649f897b13ac7bdf6be3b Jul 05 10:30:40.088330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-TestVolumeBootPattern-620052980-project Jul 05 10:30:40.088330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:30:40.088330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:30:40.088330 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': '1eeacca5-4921-455f-80e9-acee90ae762f'} Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:30:40.089341 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:30:40.090935 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:30:40.092604 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0f1f2d5c-d647-40fc-9075-552751be1473 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Returning 404 to user: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:30:40.092604 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0f1f2d5c-d647-40fc-9075-552751be1473 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "DELETE /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 404 len: 111 microversion: 2.1 time: 0.030462 Jul 05 10:30:40.092604 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 566/1160] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:30:40 2022] DELETE /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:30:40.096753 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-02c18a03-d880-4eb7-a627-a880d9be8050 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:40.097362 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-02c18a03-d880-4eb7-a627-a880d9be8050 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] [instance: 1eeacca5-4921-455f-80e9-acee90ae762f] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:30:40.103294 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-02c18a03-d880-4eb7-a627-a880d9be8050 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:40.103742 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-02c18a03-d880-4eb7-a627-a880d9be8050 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:40.105034 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-02c18a03-d880-4eb7-a627-a880d9be8050 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:30:40.105474 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-02c18a03-d880-4eb7-a627-a880d9be8050 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:30:40.118479 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-02c18a03-d880-4eb7-a627-a880d9be8050 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] HTTP exception thrown: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-02c18a03-d880-4eb7-a627-a880d9be8050 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Request method failure captured: Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f HTTP/1.1 Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixBDpuHveZ3XeN9U67EK4lKdYSp_m_64mU-LBfdZJwGXE8VS4y9KkhlpV-sF73TZRPltmWaOCmj1JJgwzGBK1g-a6vXND_jKDWiyWKsXT9eD5CtGrsU53oavQhY5y4SU9PBsCzPb3si94ZYUpzEQAVskEwEVkAMCsD3v3D2ZU1cRNTYA Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 86eac45249ad4a5e9513fcc091bcf42e Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-TestVolumeBootPattern-620052980 Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:30:40.119967 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:30:40.122319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/86eac45249ad4a5e9513fcc091bcf42e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/86eac45249ad4a5e9513fcc091bcf42e"}]}] Jul 05 10:30:40.122319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-TestVolumeBootPattern-620052980 Jul 05 10:30:40.122319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 86eac45249ad4a5e9513fcc091bcf42e Jul 05 10:30:40.122319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-TestVolumeBootPattern-620052980 Jul 05 10:30:40.122319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-TestVolumeBootPattern-620052980-project Jul 05 10:30:40.122319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:30:40.122319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:30:40.122319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: c9203890ad7649f897b13ac7bdf6be3b Jul 05 10:30:40.122319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-TestVolumeBootPattern-620052980-project Jul 05 10:30:40.122319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:30:40.122319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:30:40.122319 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': '1eeacca5-4921-455f-80e9-acee90ae762f'} Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:30:40.126071 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. Jul 05 10:30:40.127663 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:30:40.129569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-02c18a03-d880-4eb7-a627-a880d9be8050 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Returning 404 to user: Instance 1eeacca5-4921-455f-80e9-acee90ae762f could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:30:40.129569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-02c18a03-d880-4eb7-a627-a880d9be8050 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f" status: 404 len: 111 microversion: 2.1 time: 0.029052 Jul 05 10:30:40.129569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 595/1161] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:30:40 2022] GET /compute/v2.1/servers/1eeacca5-4921-455f-80e9-acee90ae762f => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:30:41.412558 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-efd3f7dd-ecf1-4a50-b058-9874b64ffa47 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] Calling method '.version_select at 0x7feb89636550>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:30:41.433785 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-efd3f7dd-ecf1-4a50-b058-9874b64ffa47 tempest-TestVolumeBootPattern-620052980 tempest-TestVolumeBootPattern-620052980-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1298886845" status: 202 len: 0 microversion: 2.1 time: 0.025734 Jul 05 10:30:41.434703 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 567/1162] 10.222.0.20 () {58 vars in 1262 bytes} [Tue Jul 5 10:30:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1298886845 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:31:40.886808 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:31:40.887536 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=8 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:31:41.050917 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-8055a156-a8cb-4115-8052-425d6c68fb12 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:31:41.051411 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-8055a156-a8cb-4115-8052-425d6c68fb12 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:31:41.057479 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8055a156-a8cb-4115-8052-425d6c68fb12 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:41.057926 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8055a156-a8cb-4115-8052-425d6c68fb12 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:41.058687 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8055a156-a8cb-4115-8052-425d6c68fb12 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:41.059151 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-8055a156-a8cb-4115-8052-425d6c68fb12 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:41.081587 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-8055a156-a8cb-4115-8052-425d6c68fb12 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/ips" status: 200 len: 146 microversion: 2.1 time: 0.196510 Jul 05 10:31:41.082128 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 596/1163] 10.222.0.20 () {58 vars in 1247 bytes} [Tue Jul 5 10:31:40 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/ips => generated 146 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:31:42.387567 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_cell1:SELECT=8 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:31:42.388568 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG dbcounter [-] [117812] Writing DB stats nova_api:SELECT=66,nova_api:DELETE=2,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=117812) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:31:42.392987 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-3bec00f5-7648-4cb2-af12-5e6e16552284 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b7702dcc-2f3e-4168-a371-9ec3f1d9929b"}} {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:31:42.394033 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-3bec00f5-7648-4cb2-af12-5e6e16552284 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:31:42.402550 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3bec00f5-7648-4cb2-af12-5e6e16552284 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:42.403004 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3bec00f5-7648-4cb2-af12-5e6e16552284 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:42.403770 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3bec00f5-7648-4cb2-af12-5e6e16552284 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:42.404206 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-3bec00f5-7648-4cb2-af12-5e6e16552284 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:42.663533 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-3bec00f5-7648-4cb2-af12-5e6e16552284 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "POST /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.278147 Jul 05 10:31:42.664134 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 568/1164] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:31:42 2022] POST /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments => generated 194 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:31:50.143949 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-247a4a03-74cb-4d9c-a29c-bc34efa2dc86 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:31:50.144960 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-247a4a03-74cb-4d9c-a29c-bc34efa2dc86 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:31:50.151494 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-247a4a03-74cb-4d9c-a29c-bc34efa2dc86 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:50.151942 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-247a4a03-74cb-4d9c-a29c-bc34efa2dc86 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:50.152726 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-247a4a03-74cb-4d9c-a29c-bc34efa2dc86 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:50.153165 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-247a4a03-74cb-4d9c-a29c-bc34efa2dc86 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:50.181622 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-247a4a03-74cb-4d9c-a29c-bc34efa2dc86 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041633 Jul 05 10:31:50.182153 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 597/1165] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:31:50 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:31:50.193506 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-0e7669e2-2362-4bc8-94e7-4ebd466346a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:31:50.194089 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-0e7669e2-2362-4bc8-94e7-4ebd466346a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:31:50.200277 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0e7669e2-2362-4bc8-94e7-4ebd466346a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:50.200781 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0e7669e2-2362-4bc8-94e7-4ebd466346a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:50.201550 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0e7669e2-2362-4bc8-94e7-4ebd466346a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:50.201996 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-0e7669e2-2362-4bc8-94e7-4ebd466346a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:50.229262 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-0e7669e2-2362-4bc8-94e7-4ebd466346a7 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments/b7702dcc-2f3e-4168-a371-9ec3f1d9929b" status: 200 len: 194 microversion: 2.1 time: 0.040609 Jul 05 10:31:50.229826 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 569/1166] 10.222.0.20 () {58 vars in 1412 bytes} [Tue Jul 5 10:31:50 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments/b7702dcc-2f3e-4168-a371-9ec3f1d9929b => generated 194 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:31:51.536286 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-484af2c1-dfbe-492a-8930-8df3c1019577 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a1c769f4-8d11-4ce3-88ea-f86ce42e6ace"}} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:31:51.537382 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-484af2c1-dfbe-492a-8930-8df3c1019577 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:31:51.543876 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-484af2c1-dfbe-492a-8930-8df3c1019577 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:51.544324 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-484af2c1-dfbe-492a-8930-8df3c1019577 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:51.545124 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-484af2c1-dfbe-492a-8930-8df3c1019577 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:51.545569 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-484af2c1-dfbe-492a-8930-8df3c1019577 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:51.793945 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-484af2c1-dfbe-492a-8930-8df3c1019577 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "POST /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.261092 Jul 05 10:31:51.794558 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 598/1167] 10.222.0.20 () {60 vars in 1322 bytes} [Tue Jul 5 10:31:51 2022] POST /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments => generated 194 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:31:56.415062 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:31:56.415718 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Tue Jul 5 10:31:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:31:59.265943 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-7564dda2-8d8d-4f4b-8a79-29a66e148ced tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:31:59.266919 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-7564dda2-8d8d-4f4b-8a79-29a66e148ced tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:31:59.273357 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7564dda2-8d8d-4f4b-8a79-29a66e148ced tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:59.273800 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7564dda2-8d8d-4f4b-8a79-29a66e148ced tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:59.274556 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7564dda2-8d8d-4f4b-8a79-29a66e148ced tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:59.274988 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-7564dda2-8d8d-4f4b-8a79-29a66e148ced tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:59.306030 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-7564dda2-8d8d-4f4b-8a79-29a66e148ced tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.043985 Jul 05 10:31:59.306757 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 570/1168] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:31:59 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments => generated 371 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:31:59.444525 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-5928f303-faf5-4568-928a-a8b3b7c39274 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:31:59.445092 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-5928f303-faf5-4568-928a-a8b3b7c39274 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:31:59.451721 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5928f303-faf5-4568-928a-a8b3b7c39274 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:59.452163 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5928f303-faf5-4568-928a-a8b3b7c39274 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:59.452943 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5928f303-faf5-4568-928a-a8b3b7c39274 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:59.453376 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-5928f303-faf5-4568-928a-a8b3b7c39274 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:59.482188 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-5928f303-faf5-4568-928a-a8b3b7c39274 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments/b7702dcc-2f3e-4168-a371-9ec3f1d9929b" status: 200 len: 194 microversion: 2.1 time: 0.041997 Jul 05 10:31:59.482740 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 599/1169] 10.222.0.20 () {58 vars in 1412 bytes} [Tue Jul 5 10:31:59 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments/b7702dcc-2f3e-4168-a371-9ec3f1d9929b => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:31:59.492680 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-cd601dcf-27d7-4b46-9ed4-63bd60185892 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:31:59.493165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-cd601dcf-27d7-4b46-9ed4-63bd60185892 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:31:59.499024 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cd601dcf-27d7-4b46-9ed4-63bd60185892 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:59.499468 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cd601dcf-27d7-4b46-9ed4-63bd60185892 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:59.500220 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cd601dcf-27d7-4b46-9ed4-63bd60185892 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:31:59.500689 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-cd601dcf-27d7-4b46-9ed4-63bd60185892 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:31:59.579944 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-cd601dcf-27d7-4b46-9ed4-63bd60185892 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lazy-loading 'services' on Instance uuid bc953afb-2d84-4da4-9478-21f39ca83773 {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:31:59.649875 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-cd601dcf-27d7-4b46-9ed4-63bd60185892 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "DELETE /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments/b7702dcc-2f3e-4168-a371-9ec3f1d9929b" status: 202 len: 0 microversion: 2.1 time: 0.160942 Jul 05 10:31:59.650809 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 571/1170] 10.222.0.20 () {58 vars in 1415 bytes} [Tue Jul 5 10:31:59 2022] DELETE /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments/b7702dcc-2f3e-4168-a371-9ec3f1d9929b => generated 0 bytes in 163 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:32:00.760475 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-429260cf-279a-4e64-9490-5be38cf7792b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:00.761028 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-429260cf-279a-4e64-9490-5be38cf7792b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:00.767260 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-429260cf-279a-4e64-9490-5be38cf7792b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:00.767706 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-429260cf-279a-4e64-9490-5be38cf7792b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:00.768500 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-429260cf-279a-4e64-9490-5be38cf7792b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:00.768938 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-429260cf-279a-4e64-9490-5be38cf7792b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:00.797989 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-429260cf-279a-4e64-9490-5be38cf7792b tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments/a1c769f4-8d11-4ce3-88ea-f86ce42e6ace" status: 200 len: 194 microversion: 2.1 time: 0.041539 Jul 05 10:32:00.798767 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 600/1171] 10.222.0.20 () {58 vars in 1412 bytes} [Tue Jul 5 10:32:00 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments/a1c769f4-8d11-4ce3-88ea-f86ce42e6ace => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:32:00.808698 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-dececf1a-e13b-4e13-833d-80700368c68d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:00.809275 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-dececf1a-e13b-4e13-833d-80700368c68d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:00.816711 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dececf1a-e13b-4e13-833d-80700368c68d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:00.817165 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dececf1a-e13b-4e13-833d-80700368c68d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:00.817937 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dececf1a-e13b-4e13-833d-80700368c68d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:00.818380 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-dececf1a-e13b-4e13-833d-80700368c68d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:00.896240 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-dececf1a-e13b-4e13-833d-80700368c68d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lazy-loading 'services' on Instance uuid bc953afb-2d84-4da4-9478-21f39ca83773 {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:32:00.965865 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-dececf1a-e13b-4e13-833d-80700368c68d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "DELETE /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments/a1c769f4-8d11-4ce3-88ea-f86ce42e6ace" status: 202 len: 0 microversion: 2.1 time: 0.161166 Jul 05 10:32:00.966455 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 572/1172] 10.222.0.20 () {58 vars in 1415 bytes} [Tue Jul 5 10:32:00 2022] DELETE /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments/a1c769f4-8d11-4ce3-88ea-f86ce42e6ace => generated 0 bytes in 163 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:32:02.122168 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-a30769b8-6923-430b-9037-f5d2bb7e56bd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:02.123089 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-a30769b8-6923-430b-9037-f5d2bb7e56bd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:02.130182 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a30769b8-6923-430b-9037-f5d2bb7e56bd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:02.130653 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a30769b8-6923-430b-9037-f5d2bb7e56bd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:02.131408 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a30769b8-6923-430b-9037-f5d2bb7e56bd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:02.131838 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-a30769b8-6923-430b-9037-f5d2bb7e56bd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:02.159520 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-a30769b8-6923-430b-9037-f5d2bb7e56bd tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041214 Jul 05 10:32:02.160048 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 601/1173] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:32:02 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments => generated 25 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 05 10:32:02.262023 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-51b3d08f-fedb-4c11-8432-3a9d62f3423c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:02.263023 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-51b3d08f-fedb-4c11-8432-3a9d62f3423c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:02.269218 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-51b3d08f-fedb-4c11-8432-3a9d62f3423c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:02.269676 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-51b3d08f-fedb-4c11-8432-3a9d62f3423c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:02.270440 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-51b3d08f-fedb-4c11-8432-3a9d62f3423c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:02.270883 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-51b3d08f-fedb-4c11-8432-3a9d62f3423c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:02.299095 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-51b3d08f-fedb-4c11-8432-3a9d62f3423c tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041989 Jul 05 10:32:02.299590 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 573/1174] 10.222.0.20 () {58 vars in 1301 bytes} [Tue Jul 5 10:32:02 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 05 10:32:02.354049 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-7cd5e56a-5e0b-4652-a82a-335dca1123c9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:02.354542 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7cd5e56a-5e0b-4652-a82a-335dca1123c9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:02.360929 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7cd5e56a-5e0b-4652-a82a-335dca1123c9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:02.361373 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7cd5e56a-5e0b-4652-a82a-335dca1123c9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:02.362122 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7cd5e56a-5e0b-4652-a82a-335dca1123c9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:02.362550 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-7cd5e56a-5e0b-4652-a82a-335dca1123c9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:02.386864 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-7cd5e56a-5e0b-4652-a82a-335dca1123c9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Going to try to terminate instance {{(pid=117813) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 05 10:32:02.457311 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-7cd5e56a-5e0b-4652-a82a-335dca1123c9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "DELETE /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 204 len: 0 microversion: 2.1 time: 0.106764 Jul 05 10:32:02.458015 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 602/1175] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:32:02 2022] DELETE /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 05 10:32:02.467980 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-40be2193-622c-40c9-8ad4-ef962748a4af tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:02.468615 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-40be2193-622c-40c9-8ad4-ef962748a4af tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:02.474464 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-40be2193-622c-40c9-8ad4-ef962748a4af tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:02.474907 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-40be2193-622c-40c9-8ad4-ef962748a4af tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:02.475663 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-40be2193-622c-40c9-8ad4-ef962748a4af tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:02.476096 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-40be2193-622c-40c9-8ad4-ef962748a4af tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:02.605361 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-40be2193-622c-40c9-8ad4-ef962748a4af tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:32:02.613316 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-40be2193-622c-40c9-8ad4-ef962748a4af tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1673 microversion: 2.1 time: 0.151008 Jul 05 10:32:02.614086 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 574/1176] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:32:02 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1673 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:32:03.631365 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-6ea48f57-7461-4a03-8dc0-17819bf70cd9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:03.631994 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-6ea48f57-7461-4a03-8dc0-17819bf70cd9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:03.638477 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6ea48f57-7461-4a03-8dc0-17819bf70cd9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:03.638926 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6ea48f57-7461-4a03-8dc0-17819bf70cd9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:03.639681 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6ea48f57-7461-4a03-8dc0-17819bf70cd9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:03.640117 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-6ea48f57-7461-4a03-8dc0-17819bf70cd9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:03.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.policy [None req-6ea48f57-7461-4a03-8dc0-17819bf70cd9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117813) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:32:03.774906 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-6ea48f57-7461-4a03-8dc0-17819bf70cd9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1673 microversion: 2.1 time: 0.147872 Jul 05 10:32:03.775459 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 603/1177] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:32:03 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1673 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:32:04.792067 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-27af0065-f8ad-4e96-9b51-6e319d836790 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:04.792700 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-27af0065-f8ad-4e96-9b51-6e319d836790 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:04.799227 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-27af0065-f8ad-4e96-9b51-6e319d836790 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:04.799675 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-27af0065-f8ad-4e96-9b51-6e319d836790 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:04.800456 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-27af0065-f8ad-4e96-9b51-6e319d836790 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:04.800896 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-27af0065-f8ad-4e96-9b51-6e319d836790 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:05.025218 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-27af0065-f8ad-4e96-9b51-6e319d836790 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:32:05.033051 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-27af0065-f8ad-4e96-9b51-6e319d836790 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1673 microversion: 2.1 time: 0.244791 Jul 05 10:32:05.033616 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 575/1178] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:32:04 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1673 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:32:05.571924 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [req-609cf14c-0420-402e-ad19-869c2ddbfc49 req-1c279097-5fe1-49d3-b831-754964a0a5d5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bc953afb-2d84-4da4-9478-21f39ca83773", "name": "network-vif-deleted", "tag": "8ccb0cb8-059d-462f-8560-f28cdf748acd"}]} {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 05 10:32:05.581222 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-609cf14c-0420-402e-ad19-869c2ddbfc49 req-1c279097-5fe1-49d3-b831-754964a0a5d5 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:05.581662 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [req-609cf14c-0420-402e-ad19-869c2ddbfc49 req-1c279097-5fe1-49d3-b831-754964a0a5d5 service nova] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:05.593207 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.compute.server_external_events [req-609cf14c-0420-402e-ad19-869c2ddbfc49 req-1c279097-5fe1-49d3-b831-754964a0a5d5 service nova] Creating event network-vif-deleted:8ccb0cb8-059d-462f-8560-f28cdf748acd for instance bc953afb-2d84-4da4-9478-21f39ca83773 on hv-cinder-maste Jul 05 10:32:05.598231 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [req-609cf14c-0420-402e-ad19-869c2ddbfc49 req-1c279097-5fe1-49d3-b831-754964a0a5d5 service nova] 10.222.0.20 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029792 Jul 05 10:32:05.598742 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 604/1179] 10.222.0.20 () {64 vars in 1317 bytes} [Tue Jul 5 10:32:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 05 10:32:06.051077 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c1016ee5-9498-4465-a460-07daa0f9823d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:06.052140 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-c1016ee5-9498-4465-a460-07daa0f9823d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:06.059862 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c1016ee5-9498-4465-a460-07daa0f9823d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:06.060368 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c1016ee5-9498-4465-a460-07daa0f9823d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:06.061229 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c1016ee5-9498-4465-a460-07daa0f9823d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:06.062056 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c1016ee5-9498-4465-a460-07daa0f9823d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:06.103011 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c1016ee5-9498-4465-a460-07daa0f9823d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:06.103796 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c1016ee5-9498-4465-a460-07daa0f9823d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:06.104428 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.objects.instance [None req-c1016ee5-9498-4465-a460-07daa0f9823d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lazy-loading 'fault' on Instance uuid bc953afb-2d84-4da4-9478-21f39ca83773 {{(pid=117812) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 05 10:32:06.149337 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.policy [None req-c1016ee5-9498-4465-a460-07daa0f9823d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b131286ad9144575a13972e25cab20db', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b5c262b91db42cb9e4042b180f9fb4d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117812) authorize /opt/stack/nova/nova/policy.py:199}} Jul 05 10:32:06.157064 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-c1016ee5-9498-4465-a460-07daa0f9823d tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 200 len: 1327 microversion: 2.1 time: 0.110219 Jul 05 10:32:06.157639 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 576/1180] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:32:06 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 1327 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 05 10:32:07.174852 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e6b48f5f-fe24-46ff-8bdf-40ba41ff3689 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:07.175490 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-e6b48f5f-fe24-46ff-8bdf-40ba41ff3689 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:07.181621 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e6b48f5f-fe24-46ff-8bdf-40ba41ff3689 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:07.182061 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e6b48f5f-fe24-46ff-8bdf-40ba41ff3689 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:07.182811 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e6b48f5f-fe24-46ff-8bdf-40ba41ff3689 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:07.183240 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-e6b48f5f-fe24-46ff-8bdf-40ba41ff3689 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:07.196022 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-e6b48f5f-fe24-46ff-8bdf-40ba41ff3689 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] HTTP exception thrown: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e6b48f5f-fe24-46ff-8bdf-40ba41ff3689 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Request method failure captured: Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 HTTP/1.1 Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixBGxtukDdXChxNGcBf9KIYDoYTTOhMKKx8ma5aSkC_edsEMwgjZenXD_0fRHl_3y99r5Di-3mSh8hvcSudbvKqdkmjXW-qUepUKEtxJjiRlFxE4KvwEcYupkYfYyEbI7BcaMd5v0JHgQRSF1xD4XX5OzLrDMw1GdWjNZlwRN9Y6Rmsk Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 8b5c262b91db42cb9e4042b180f9fb4d Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:32:07.197630 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:32:07.199384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/8b5c262b91db42cb9e4042b180f9fb4d"}]}] Jul 05 10:32:07.199384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:07.199384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 8b5c262b91db42cb9e4042b180f9fb4d Jul 05 10:32:07.199384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:07.199384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeTestJSON-1564756241-project Jul 05 10:32:07.199384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:32:07.199384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:32:07.199384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: b131286ad9144575a13972e25cab20db Jul 05 10:32:07.199384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeTestJSON-1564756241-project Jul 05 10:32:07.199384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:32:07.199384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:07.199384 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'bc953afb-2d84-4da4-9478-21f39ca83773'} Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:32:07.200427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:07.202085 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:07.203696 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-e6b48f5f-fe24-46ff-8bdf-40ba41ff3689 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Returning 404 to user: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:32:07.203696 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-e6b48f5f-fe24-46ff-8bdf-40ba41ff3689 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 404 len: 111 microversion: 2.1 time: 0.029418 Jul 05 10:32:07.203696 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 605/1181] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:32:07 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:32:07.207657 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-ef6f0d5a-da42-4778-ae66-5c7f755ee94a tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '.version_select at 0x7feb898618b0>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:07.226197 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-ef6f0d5a-da42-4778-ae66-5c7f755ee94a tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1316808162" status: 202 len: 0 microversion: 2.1 time: 0.022218 Jul 05 10:32:07.227073 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 577/1182] 10.222.0.20 () {58 vars in 1220 bytes} [Tue Jul 5 10:32:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1316808162 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 05 10:32:10.075786 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-74f7e871-1c5b-4f20-bad9-7640ccc672bb tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:10.076302 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-74f7e871-1c5b-4f20-bad9-7640ccc672bb tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:10.083066 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-74f7e871-1c5b-4f20-bad9-7640ccc672bb tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:10.083510 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-74f7e871-1c5b-4f20-bad9-7640ccc672bb tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:10.084259 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-74f7e871-1c5b-4f20-bad9-7640ccc672bb tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:10.084788 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-74f7e871-1c5b-4f20-bad9-7640ccc672bb tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:10.098271 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-74f7e871-1c5b-4f20-bad9-7640ccc672bb tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] HTTP exception thrown: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-74f7e871-1c5b-4f20-bad9-7640ccc672bb tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Request method failure captured: Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: DELETE /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 HTTP/1.1 Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixBGxtukDdXChxNGcBf9KIYDoYTTOhMKKx8ma5aSkC_edsEMwgjZenXD_0fRHl_3y99r5Di-3mSh8hvcSudbvKqdkmjXW-qUepUKEtxJjiRlFxE4KvwEcYupkYfYyEbI7BcaMd5v0JHgQRSF1xD4XX5OzLrDMw1GdWjNZlwRN9Y6Rmsk Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 8b5c262b91db42cb9e4042b180f9fb4d Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:32:10.099815 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:32:10.101541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/8b5c262b91db42cb9e4042b180f9fb4d"}]}] Jul 05 10:32:10.101541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:10.101541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 8b5c262b91db42cb9e4042b180f9fb4d Jul 05 10:32:10.101541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:10.101541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeTestJSON-1564756241-project Jul 05 10:32:10.101541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:32:10.101541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:32:10.101541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: b131286ad9144575a13972e25cab20db Jul 05 10:32:10.101541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeTestJSON-1564756241-project Jul 05 10:32:10.101541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:32:10.101541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:10.101541 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'bc953afb-2d84-4da4-9478-21f39ca83773'} Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:32:10.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:10.104169 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:10.105753 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-74f7e871-1c5b-4f20-bad9-7640ccc672bb tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Returning 404 to user: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:32:10.105753 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-74f7e871-1c5b-4f20-bad9-7640ccc672bb tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "DELETE /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 404 len: 111 microversion: 2.1 time: 0.029289 Jul 05 10:32:10.105753 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 606/1183] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:32:10 2022] DELETE /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:32:10.110858 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c07f8126-175b-4a0f-80ed-1fa6230e0d04 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:10.111473 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-c07f8126-175b-4a0f-80ed-1fa6230e0d04 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: bc953afb-2d84-4da4-9478-21f39ca83773] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:10.121123 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c07f8126-175b-4a0f-80ed-1fa6230e0d04 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:10.121577 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c07f8126-175b-4a0f-80ed-1fa6230e0d04 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:10.122360 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c07f8126-175b-4a0f-80ed-1fa6230e0d04 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:10.122798 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-c07f8126-175b-4a0f-80ed-1fa6230e0d04 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:10.135763 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-c07f8126-175b-4a0f-80ed-1fa6230e0d04 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] HTTP exception thrown: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c07f8126-175b-4a0f-80ed-1fa6230e0d04 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Request method failure captured: Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 HTTP/1.1 Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixBGxtukDdXChxNGcBf9KIYDoYTTOhMKKx8ma5aSkC_edsEMwgjZenXD_0fRHl_3y99r5Di-3mSh8hvcSudbvKqdkmjXW-qUepUKEtxJjiRlFxE4KvwEcYupkYfYyEbI7BcaMd5v0JHgQRSF1xD4XX5OzLrDMw1GdWjNZlwRN9Y6Rmsk Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: 8b5c262b91db42cb9e4042b180f9fb4d Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:32:10.137421 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:32:10.139090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/8b5c262b91db42cb9e4042b180f9fb4d"}]}] Jul 05 10:32:10.139090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:10.139090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: 8b5c262b91db42cb9e4042b180f9fb4d Jul 05 10:32:10.139090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:10.139090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-AttachVolumeTestJSON-1564756241-project Jul 05 10:32:10.139090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:32:10.139090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:32:10.139090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: b131286ad9144575a13972e25cab20db Jul 05 10:32:10.139090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-AttachVolumeTestJSON-1564756241-project Jul 05 10:32:10.139090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:32:10.139090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:10.139090 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': 'bc953afb-2d84-4da4-9478-21f39ca83773'} Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:32:10.140116 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. Jul 05 10:32:10.141812 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:32:10.143332 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-c07f8126-175b-4a0f-80ed-1fa6230e0d04 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Returning 404 to user: Instance bc953afb-2d84-4da4-9478-21f39ca83773 could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:32:10.143332 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-c07f8126-175b-4a0f-80ed-1fa6230e0d04 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773" status: 404 len: 111 microversion: 2.1 time: 0.033207 Jul 05 10:32:10.143332 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 578/1184] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:32:10 2022] GET /compute/v2.1/servers/bc953afb-2d84-4da4-9478-21f39ca83773 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:32:11.270144 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d093dcb2-d786-45c7-88cc-750c5c14dc41 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:11.270648 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.compute.api [None req-d093dcb2-d786-45c7-88cc-750c5c14dc41 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117813) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:11.273035 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_api:SELECT=28,nova_api:UPDATE=1 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:32:11.278497 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d093dcb2-d786-45c7-88cc-750c5c14dc41 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:11.278944 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d093dcb2-d786-45c7-88cc-750c5c14dc41 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:11.279706 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d093dcb2-d786-45c7-88cc-750c5c14dc41 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:11.280148 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG oslo_concurrency.lockutils [None req-d093dcb2-d786-45c7-88cc-750c5c14dc41 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117813) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:11.284679 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG dbcounter [-] [117813] Writing DB stats nova_cell1:SELECT=66,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117813) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 10:32:11.297999 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.wsgi [None req-d093dcb2-d786-45c7-88cc-750c5c14dc41 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] HTTP exception thrown: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d093dcb2-d786-45c7-88cc-750c5c14dc41 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Request method failure captured: Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: request: DELETE /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c HTTP/1.1 Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept: application/json Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Accept-Encoding: identity Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Connection: close Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Length: 0 Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Content-Type: application/json Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Host: 10.222.0.20 Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Openstack-System-Scope: None Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: User-Agent: python-urllib3/1.26.9 Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Auth-Token: gAAAAABixBGxtukDdXChxNGcBf9KIYDoYTTOhMKKx8ma5aSkC_edsEMwgjZenXD_0fRHl_3y99r5Di-3mSh8hvcSudbvKqdkmjXW-qUepUKEtxJjiRlFxE4KvwEcYupkYfYyEbI7BcaMd5v0JHgQRSF1xD4XX5OzLrDMw1GdWjNZlwRN9Y6Rmsk Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Id: None Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Domain-Name: None Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Identity-Status: Confirmed Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Is-Admin-Project: True Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Id: default Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Domain-Name: Default Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Id: 8b5c262b91db42cb9e4042b180f9fb4d Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Project-Name: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Role: member,reader Jul 05 10:32:11.299734 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Roles: member,reader Jul 05 10:32:11.301443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/8b5c262b91db42cb9e4042b180f9fb4d"}]}] Jul 05 10:32:11.301443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:11.301443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Id: 8b5c262b91db42cb9e4042b180f9fb4d Jul 05 10:32:11.301443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:11.301443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User: tempest-AttachVolumeTestJSON-1564756241-project Jul 05 10:32:11.301443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Id: default Jul 05 10:32:11.301443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Domain-Name: Default Jul 05 10:32:11.301443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Id: b131286ad9144575a13972e25cab20db Jul 05 10:32:11.301443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: X-User-Name: tempest-AttachVolumeTestJSON-1564756241-project Jul 05 10:32:11.301443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: method: > Jul 05 10:32:11.301443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: exception: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:32:11.301443 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: action_args: {'id': 'a65a032c-5b68-4e65-b02c-d76e6547c26c'} Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: {{(pid=117813) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:32:11.302385 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:32:11.303941 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: ERROR nova.api.openstack.wsgi Jul 05 10:32:11.305652 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: DEBUG nova.api.openstack.wsgi [None req-d093dcb2-d786-45c7-88cc-750c5c14dc41 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Returning 404 to user: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. {{(pid=117813) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:32:11.305652 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO nova.api.openstack.requestlog [None req-d093dcb2-d786-45c7-88cc-750c5c14dc41 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "DELETE /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 404 len: 111 microversion: 2.1 time: 0.034924 Jul 05 10:32:11.305652 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: [pid: 117813|app: 0|req: 607/1185] 10.222.0.20 () {58 vars in 1238 bytes} [Tue Jul 5 10:32:11 2022] DELETE /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:32:11.309475 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-558cd896-19e5-4bdf-984d-f8ff4623f0f9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Calling method '>' {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 05 10:32:11.310085 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.compute.api [None req-558cd896-19e5-4bdf-984d-f8ff4623f0f9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] [instance: a65a032c-5b68-4e65-b02c-d76e6547c26c] Fetching instance by UUID {{(pid=117812) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 05 10:32:11.316501 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-558cd896-19e5-4bdf-984d-f8ff4623f0f9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:11.316952 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-558cd896-19e5-4bdf-984d-f8ff4623f0f9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:11.317714 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-558cd896-19e5-4bdf-984d-f8ff4623f0f9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 05 10:32:11.318152 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG oslo_concurrency.lockutils [None req-558cd896-19e5-4bdf-984d-f8ff4623f0f9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Lock "21c70492-35f8-4468-a997-987c705bd844" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117812) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 05 10:32:11.331531 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.wsgi [None req-558cd896-19e5-4bdf-984d-f8ff4623f0f9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] HTTP exception thrown: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-558cd896-19e5-4bdf-984d-f8ff4623f0f9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Request method failure captured: Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: request: GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c HTTP/1.1 Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept: application/json Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Accept-Encoding: identity Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Connection: close Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Length: 0 Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Content-Type: application/json Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Host: 10.222.0.20 Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Openstack-System-Scope: None Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: User-Agent: python-urllib3/1.26.9 Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Auth-Token: gAAAAABixBGxtukDdXChxNGcBf9KIYDoYTTOhMKKx8ma5aSkC_edsEMwgjZenXD_0fRHl_3y99r5Di-3mSh8hvcSudbvKqdkmjXW-qUepUKEtxJjiRlFxE4KvwEcYupkYfYyEbI7BcaMd5v0JHgQRSF1xD4XX5OzLrDMw1GdWjNZlwRN9Y6Rmsk Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Id: None Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Domain-Name: None Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Identity-Status: Confirmed Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Is-Admin-Project: True Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Id: default Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Domain-Name: Default Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Id: 8b5c262b91db42cb9e4042b180f9fb4d Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Project-Name: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Role: member,reader Jul 05 10:32:11.333050 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Roles: member,reader Jul 05 10:32:11.334784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20:9696/"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2/8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.20:8080", "publicURL": "http://10.222.0.20:8080/v1/AUTH_8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/8b5c262b91db42cb9e4042b180f9fb4d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.20/volume/v3/8b5c262b91db42cb9e4042b180f9fb4d"}]}] Jul 05 10:32:11.334784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:11.334784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Id: 8b5c262b91db42cb9e4042b180f9fb4d Jul 05 10:32:11.334784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1564756241 Jul 05 10:32:11.334784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User: tempest-AttachVolumeTestJSON-1564756241-project Jul 05 10:32:11.334784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Id: default Jul 05 10:32:11.334784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Domain-Name: Default Jul 05 10:32:11.334784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Id: b131286ad9144575a13972e25cab20db Jul 05 10:32:11.334784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: X-User-Name: tempest-AttachVolumeTestJSON-1564756241-project Jul 05 10:32:11.334784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: method: > Jul 05 10:32:11.334784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: exception: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:32:11.334784 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: action_args: {'id': 'a65a032c-5b68-4e65-b02c-d76e6547c26c'} Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: {{(pid=117812) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 05 10:32:11.335745 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. Jul 05 10:32:11.337317 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: ERROR nova.api.openstack.wsgi Jul 05 10:32:11.338696 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: DEBUG nova.api.openstack.wsgi [None req-558cd896-19e5-4bdf-984d-f8ff4623f0f9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] Returning 404 to user: Instance a65a032c-5b68-4e65-b02c-d76e6547c26c could not be found. {{(pid=117812) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 05 10:32:11.338696 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO nova.api.openstack.requestlog [None req-558cd896-19e5-4bdf-984d-f8ff4623f0f9 tempest-AttachVolumeTestJSON-1564756241 tempest-AttachVolumeTestJSON-1564756241-project] 10.222.0.20 "GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c" status: 404 len: 111 microversion: 2.1 time: 0.028725 Jul 05 10:32:11.338696 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: [pid: 117812|app: 0|req: 579/1186] 10.222.0.20 () {58 vars in 1235 bytes} [Tue Jul 5 10:32:11 2022] GET /compute/v2.1/servers/a65a032c-5b68-4e65-b02c-d76e6547c26c => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 05 10:32:42.441427 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:32:42.442098 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Tue Jul 5 10:32:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:35:16.422697 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:35:16.423792 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Tue Jul 5 10:35:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:35:20.599641 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:35:20.600233 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Tue Jul 5 10:35:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:35:27.053644 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:35:27.054255 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Tue Jul 5 10:35:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:36:02.449525 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:36:02.450166 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Tue Jul 5 10:36:02 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:37:13.006475 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:37:13.007046 dv-cinder-master-3-iscsi devstack@n-api.service[117813]: Tue Jul 5 10:37:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 05 10:37:17.488029 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 05 10:37:17.488609 dv-cinder-master-3-iscsi devstack@n-api.service[117812]: Tue Jul 5 10:37:17 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!